A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.

Some implications of lepton flavor violating processes in a supersymmetric Type II seesaw model at TeV scale

We have conceived a supersymmetric Type II seesaw model at TeV scale, which has some additional particles consisting of scalar and fermionic triplet Higgs states, whose masses are around a few hundred GeV. In this particular model, we have studied constraints on the masses of triplet states arising from the lepton flavor violating (LFV) processes, such as mu -> 3e and mu -> e gamma. We have analyzed the implications of these constraints on other observable quantities such as the muon anomalous magnetic moment and the decay patterns of scalar triplet Higgses. Scalar triplet Higgs states can decay into leptons and into supersymmetric fields. We have found that the constraints from LFV can affect these various decay modes.

Flow cytometry analysis of human fetal osteoblast fate processes on spark plasma sintered hydroxyapatite-titanium biocomposites

Hydroxyapatite (HA)-based biocomposites have been widely investigated for a multitude of applications and these studies have been largely driven to improve mechanical properties (toughness and strength) without compromising cytocompatibility properties. Apart from routine cell viability/proliferation analysis, limited efforts have been made to quantify the fate processes (cell proliferation, cell cycle, and cell apoptosis) of human fetal osteoblast (hFOB) cells on HA-based composites, in vitro. In this work, the osteoblast cell fate process has been studied on a model hydroxyapatite-titanium (HA-Ti) system using the flow cytometry. In order to retain both HA and Ti, the novel processing technique, that is, spark plasma sintering, was suitably adopted. The cell fate processes of hFOBs, as evaluated using a flow cytometry, revealed statistically insignificant differences among HA-10 wt % Ti and HA and control (tissue culture polystyrene surface) in terms of osteoblast apoptosis, proliferation index as well as division index. For the first time, we provide quantified flow cytometry results to demonstrate that 10 wt % Ti additions to HA do not have any significant influence on the fate processes of human osteoblast-like cells, in vitro.

Validation based sparse Gaussian processes for ordinal regression

This paper proposes a sparse modeling approach to solve ordinal regression problems using Gaussian processes (GP). Designing a sparse GP model is important from training time and inference time viewpoints. We first propose a variant of the Gaussian process ordinal regression (GPOR) approach, leave-one-out GPOR (LOO-GPOR). It performs model selection using the leave-one-out cross-validation (LOO-CV) technique. We then provide an approach to design a sparse model for GPOR. The sparse GPOR model reduces computational time and storage requirements. Further, it provides faster inference. We compare the proposed approaches with the state-of-the-art GPOR approach on some benchmark data sets. Experimental results show that the proposed approaches are competitive.

Morphology and electrostatics play active role in neuronal differentiation processes on flexible conducting substrates

This commentary discusses and summarizes the key highlights of our recently reported work entitled ``Neuronal Differentiation of Embryonic Stem Cell Derived Neuronal Progenitors Can Be Regulated by Stretchable Conducting Polymers.'' The prospect of controlling the mechanical-rigidity and the surface conductance properties offers a unique combination for tailoring the growth and differentiation of neuronal cells. We emphasize the utility of transparent elastomeric substrates with coatings of electrically conducting polymer to realize the desired substrate-characteristics for cellular development processes. Our study showed that neuronal differentiation from ES cells is highly influenced by the specific substrates on which they are growing. Thus, our results provide a better strategy for regulated neuronal differentiation by using such functional conducting surfaces.

Imprints of Archean to Neoproterozoic crustal processes in the Madurai Block, Southern India

Madurai Block, the largest crustal block in the Southern Granulite Terrane (SGT) of Peninsular India, preserves the imprints of multistage tectonic evolution. Here, we present U-Pb and Hf isotope data on zircons from a charnockite-granite suite in the north-western part of this block. The oscillatory zoning, and the LREE to HREE enriched patterns of the zircons with positive Ce and negative Eu anomalies suggest that the zircon cores are of magmatic origin, with ages in the range of 2634-2435 Ma implying Neoarchean-Paleoproterozoic magmatism followed by subsequent metamorphism and protocontinent formation in the north-western part of the Madurai Block. A regional 550-500 Ma metamorphic overprint is also preserved in the zircons coinciding with the final amalgamation of the Gondwana supercontinent. The Hf isotopic data suggest that the granite and charnockite were derived from isotopically heterogeneous juvenile crustal domains and the charnockites show a significant contribution of mantle-derived components. Therefore, the Hf isotopic data reflect mixing of crustal and mantle-derived sources for the generation of Neoarchean crust in the north-western Madurai Block, possibly in a suprasubduction zone setting during continent building processes. (c) 2014 Elsevier Ltd. All rights reserved.

A modeling study of the processes of surface salinity seasonal cycle in the Bay of Bengal

In response to the Indian Monsoon freshwater forcing, the Bay of Bengal exhibits a very strong seasonal cycle in sea surface salinity (SSS), especially near the mouths of the Ganges-Brahmaputra and along the east coast of India. In this paper, we use an eddy-permitting (similar to 25 km resolution) regional ocean general circulation model simulation to quantify the processes responsible for this SSS seasonal cycle. Despite the absence of relaxation toward observations, the model reproduces the main features of the observed SSS seasonal cycle, with freshest water in the northeastern Bay, particularly during and after the monsoon. The model also displays an intense and shallow freshening signal in a narrow (similar to 100 km wide) strip that hugs the east coast of India, from September to January, in good agreement with high-resolution measurements along two ships of opportunity lines. The mixed layer salt budget confirms that the strong freshening in the northern Bay during the monsoon results from the Ganges-Brahmaputra river discharge and from precipitation over the ocean. From September onward, the East India Coastal Current transports this freshwater southward along the east coast of India, reaching the southern tip of India in November. The surface freshening results in an enhanced vertical salinity gradient that increases salinity of the surface layer by vertical processes. Our results reveal that the erosion of the freshwater tongue along the east coast of India is not driven by northward horizontal advection, but by vertical processes that eventually overcome the freshening by southward advection and restore SSS to its premonsoon values. The salinity-stratified barrier layer hence only acts as a ``barrier'' for vertical heat fluxes, but is associated with intense vertical salt fluxes in the Bay of Bengal.

Approximation of Spatio-Temporal Random Processes Using Tensor Decomposition

A new representation of spatio-temporal random processes is proposed in this work. In practical applications, such processes are used to model velocity fields, temperature distributions, response of vibrating systems, to name a few. Finding an efficient representation for any random process leads to encapsulation of information which makes it more convenient for a practical implementations, for instance, in a computational mechanics problem. For a single-parameter process such as spatial or temporal process, the eigenvalue decomposition of the covariance matrix leads to the well-known Karhunen-Loeve (KL) decomposition. However, for multiparameter processes such as a spatio-temporal process, the covariance function itself can be defined in multiple ways. Here the process is assumed to be measured at a finite set of spatial locations and a finite number of time instants. Then the spatial covariance matrix at different time instants are considered to define the covariance of the process. This set of square, symmetric, positive semi-definite matrices is then represented as a third-order tensor. A suitable decomposition of this tensor can identify the dominant components of the process, and these components are then used to define a closed-form representation of the process. The procedure is analogous to the KL decomposition for a single-parameter process, however, the decompositions and interpretations vary significantly. The tensor decompositions are successfully applied on (i) a heat conduction problem, (ii) a vibration problem, and (iii) a covariance function taken from the literature that was fitted to model a measured wind velocity data. It is observed that the proposed representation provides an efficient approximation to some processes. Furthermore, a comparison with KL decomposition showed that the proposed method is computationally cheaper than the KL, both in terms of computer memory and execution time.

Breakup processes of pressure swirl spray in gaseous cross-flow

Injection of liquid fuel in cross flowing air has been a strategy for future aircraft engines in order to control the emissions. In this context, breakup of a pressure swirl spray in gaseous cross-flow is investigated experimentally. The atomizer discharges a conical swirling sheet of liquid that interacts with cross-flowing air. This complex interaction and the resulting spray structures at various flow conditions are studied through flow visualization using still as well as high speed photography. Experiments are performed over a wide range of aerodynamic Weber number (2-300) and liquid-to-air momentum flux ratio (5-150). Various breakup regimes exhibiting different breakup processes are mapped on a parameter space based on flow conditions. This map shows significant variations from breakup regime map for a plain liquid jet in cross-flow. It is observed that the breakup of leeward side of the sheet is dominated by bag breakup and the windward side of the sheet undergoes breakup through surface waves. Similarities and differences between bag breakup present in plain liquid jet in cross-flow and swirl spray in cross-flow are explained. Multimodal drop size distribution from bag breakup, frequency of bag breakup, wavelength of surface waves and trajectory of spray in cross-flow are measured by analyzing the spray images and parametric study of their variations is also presented. (C) 2014 Elsevier Ltd. All rights reserved.

Sources of major ions and processes affecting the geochemical and isotopic signatures of subsurface waters along a tropical river, Southwestern India

Systematic monitoring of subsurface hydrogeochemistry has been carried out for a period of one year in a humid tropical region along the Nethravati-Gurupur River. The major ion and stable isotope (delta O-18 and delta H-2) compositions are used to understand the hydrogeochemistry of groundwater and its interaction with surface water. In the study, it is observed that intense weathering of source rocks is the major source of chemical elements to the surface and subsurface waters. In addition, agricultural activities and atmospheric contributions also control the major ion chemistry of water in the study area. There is a clear seasonality in the groundwater chemistry, which is related to the recharge and discharge of the hydrological system. On a temporal scale, there is a decrease in major cation concentrations during the monsoon which is a result of dilution of sources from the weathering of rock minerals, and an increase in anion concentrations which is contributed by the atmosphere, accompanied by an increase in water level during the monsoon. The stable isotope composition indicates that groundwater in the basin is of meteoric origin and recharged directly from the local precipitation during the monsoonal season. Soon after the monsoon, groundwater and surface water mix in the subsurface region. The groundwater feeds the surface water during the lean river flow season.

Investigation on two magnon scattering processes in pulsed laser deposited epitaxial nickel zinc ferrite thin film

Ferromagnetic resonance (FMR) measurements are employed to evaluate the presence of the two magnon scattering contribution in the magnetic relaxation processes of the epitaxial nickel zinc ferrite thin films deposited using pulsed laser deposition (PLD) on the (0 0 1) MgAl2O4 substrate. Furthermore, the reciprocal space mapping reveals the presence of microstructural defects which acts as an origin for the two magnon scattering process in this thin film. The relevance of this scattering process is further discussed for understanding the higher FMR linewidth in the in-plane configuration compared to the out-of-plane configuration. FMR measurements also reveal the presence of competing uniaxial and cubic anisotropy in the studied films.

Risk-Sensitive Ergodic Control of Continuous Time Markov Processes With Denumerable State Space

In this article, we study risk-sensitive control problem with controlled continuous time Markov chain state dynamics. Using multiplicative dynamic programming principle along with the atomic structure of the state dynamics, we prove the existence and a characterization of optimal risk-sensitive control under geometric ergodicity of the state dynamics along with a smallness condition on the running cost.

The electronic, chemical and electrocatalytic processes and intermediates on iron oxide surfaces during photoelectrochemical water splitting

We re-assess experimental soft X-ray absorption spectra of the oxygen K-shell which we recorded operando from iron oxide during photoelectrochemical water splitting in KOH electrolyte. In particular, we refer to recently reported transitional electron hole states which originate within the charge carrier depletion layer of the iron oxide and on the iron oxide surface. For the latter we find that an intermediate oxy-peroxo species is formed on the iron oxide with increasing bias potential, which disappears upon further polarization of the electrode, concomitantly with the evolution and disappearance of the aforementioned surface state. The oxygen spectra contain also the spectroscopic signatures of the electrolyte water, the position of which changes with increasing bias potential towards lower X-ray energies, revealing the breaking and formation of hydrogen bonds in the water during the experiment. Combined with potential dependent impedance spectroscopy data we are able to sketch the molecular structure of chemical intermediates and their charge carrier dynamics. (C) 2015 Elsevier B.V. All rights reserved.

Use of Sr isotopes as a tool to decipher the soil weathering processes in a tropical river catchment, southwestern India

River water composition (major ion and Sr-87/Sr-86 ratio) was monitored on a monthly basis over a period of three years from a mountainous river (Nethravati River) of southwestern India. The total dissolved solid (TDS) concentration is relatively low (46 mg L-1) with silica being the dominant contributor. The basin is characterised by lower dissolved Sr concentration (avg. 150 nmol L-1), with radiogenic Sr-87/Sr-86 isotopic ratios (avg. 0.72041 at outlet). The composition of Sr and Sr-87/Sr-86 and their correlation with silicate derived cations in the river basin reveal that their dominant source is from the radiogenic silicate rock minerals. Their composition in the stream is controlled by a combination of physical and chemical weathering occurring in the basin. The molar ratio of SiO2/Ca and Sr-87/Sr-86 isotopic ratio show strong seasonal variation in the river water, i.e., low SiO2/Ca ratio with radiogenic isotopes during non-monsoon and higher SiO2/Ca with less radiogenic isotopes during monsoon season. Whereas, the seasonal variation of Rb/Sr ratio in the stream water is not significant suggesting that change in the mineral phase being involved in the weathering reaction could be unlikely for the observed molar SiO2/Ca and Sr-87/Sr-86 isotope variation in river water. Therefore, the shift in the stream water chemical composition could be attributed to contribution of ground water which is in contact with the bedrock (weathering front) during non-monsoon and weathering of secondary soil minerals in the regolith layer during monsoon. The secondary soil mineral weathering leads to limited silicate cation and enhanced silica fluxes in the Nethravati river basin. (C) 2015 Elsevier Ltd. All rights reserved.