An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Simulating the daily discharge of the Mandovi River, west coast of India

A hydrological modelling framework was assembled to simulate the daily discharge of the Mandovi River on the Indian west coast. Approximately 90% of the west-coast rainfall, and therefore discharge, occurs during the summer monsoon (June-September), with a peak during July-August. The modelling framework consisted of a digital elevation model (DEM) called GLOBE, a hydrological routing algorithm, the Terrestrial Hydrological Model with Biogeochemistry (THMB), an algorithm to map the rainfall recorded by sparse rain-gauges to the model grid, and a modified Soil Conservation Service Curve Number (SCS-CN) method. A series of discharge simulations (with and without the SCS method) was carried out. The best simulation was obtained after incorporating spatio-temporal variability in the SCS parameters, which was achieved by an objective division of the season into five regimes: the lean season, monsoon onset, peak monsoon, end-monsoon, and post-monsoon. A novel attempt was made to incorporate objectively the different regimes encountered before, during and after the Indian monsoon, into a hydrological modelling framework. The strength of our method lies in the low demand it makes on hydrological data. Apart from information on the average soil type in a region, the entire parameterization is built on the basis of the rainfall that is used to force the model. That the model does not need to be calibrated separately for each river is important, because most of the Indian west-coast basins are ungauged. Hence, even though the model has been validated only for the Mandovi basin, its potential region of application is considerable. In the context of the Prediction in Ungauged Basins (PUB) framework, the potential of the proposed approach is significant, because the discharge of these (ungauged) rivers into the eastern Arabian Sea is not small, making them an important element of the local climate system.

Nanoindentation as a probe for mechanically-induced molecular migration in layered organic donor-acceptor complexes

Nanoindentation and scratch experiments on 1:1 donor-acceptor complexes, 1 and 2, of 1,2,4,5-tetracyanobenzene with pyrene and phenanthrene, respectively, reveal long-range molecular layer gliding and large interaction anisotropy. Due to the layered arrangements in these crystals, these experiments that apply stress in particular directions result in the breaking of interlayer interactions, thus allowing molecular sheets to glide over one another with ease. Complex 1 has a layered crystal packing wherein the layers are 68° skew under the (002) face and the interlayer space is stabilized by van der Waals interactions. Upon indenting this surface with a Berkovich tip, pile-up of material was observed on just one side of the indenter due to the close angular alignment of the layers with the half angle of the indenter tip (65.35°). The interfacial differences in the elastic modulus (21 ) and hardness (16 ) demonstrate the anisotropic nature of crystal packing. In 2, the molecular stacks are arranged in a staggered manner; there is no layer arrangement, and the interlayer stabilization involves C-H�N hydrogen bonds and ��� interactions. This results in a higher modulus (20 ) for (020) as compared to (001), although the anisotropy in hardness is minimal (4 ). The anisotropy within a face was analyzed using AFM image scans and the coefficient of friction of four orthogonal nanoscratches on the cleavage planes of 1 and 2. A higher friction coefficient was obtained for 2 as compared to 1 even in the cleavage direction due to the presence of hydrogen bonds in the interlayer region making the tip movement more hindered. Copyright © 2012 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Computational algorithms inspired by biological processes and evolution

In recent times computational algorithms inspired by biological processes and evolution are gaining much popularity for solving science and engineering problems. These algorithms are broadly classified into evolutionary computation and swarm intelligence algorithms, which are derived based on the analogy of natural evolution and biological activities. These include genetic algorithms, genetic programming, differential evolution, particle swarm optimization, ant colony optimization, artificial neural networks, etc. The algorithms being random-search techniques, use some heuristics to guide the search towards optimal solution and speed-up the convergence to obtain the global optimal solutions. The bio-inspired methods have several attractive features and advantages compared to conventional optimization solvers. They also facilitate the advantage of simulation and optimization environment simultaneously to solve hard-to-define (in simple expressions), real-world problems. These biologically inspired methods have provided novel ways of problem-solving for practical problems in traffic routing, networking, games, industry, robotics, economics, mechanical, chemical, electrical, civil, water resources and others fields. This article discusses the key features and development of bio-inspired computational algorithms, and their scope for application in science and engineering fields.

A new diketopyrrolopyrrole-based co-polymer for ambipolar field-effect transistors and solar cells

A new thieno3,2-b]thiophenediketopyrrolopyrrole-benzo1,2-b:4,5-b']dithio phene based narrow optical gap co-polymer (PTTDPP-BDT) has been synthesized and characterized for field-effect transistors and solar cells. In field-effect transistors the polymer exhibited ambipolar charge transport behaviour with maximum hole and electron mobilities of 10(-3) cm(2) V-1 s(-1) and 10(-5) cm(2)V(-1) s(-1), respectively. The respectable charge transporting properties of the polymer were consistent with X-ray diffraction measurements that showed close molecular packing in the solid state. The difference in hole and electron mobilities was explained by density functional theory calculations, which showed that the highest occupied molecular orbital was delocalized along the polymer backbone with the lowest unoccupied molecular orbital localized on the bis(thieno3,2-b]thiophene)diketopyrrolopyrrole units. Bulk heterojunction photovoltaic devices with the fullerene acceptor PC70BM were fabricated and delivered a maximum conversion efficiency of 3.3% under AM1.5G illumination. (C) 2012 Elsevier B.V. All rights reserved.

Small-Angle Neutron-Scattering Studies of Mixed Micellar Structures Made of Dimeric Surfactants Having Imidazolium and Ammonium Headgroups

Planar imidazolium cation based gemini surfactants 16-Im-n-Im-16], 2Br(-) (where n = 2, 3, 4, 5, 6, 8, 10, and 12), exhibit different morphologies and internal packing arrangements by adopting different supramolecular assemblies in aqueous media depending on their number of spacer methylene units (CH2)(n). Detailed measurements of the small-angle neutron-scattering (SANS) cross sections from different imidazolium-based surfactant micelles in aqueous media (D2O) are reported. The SANS data, containing the information of aggregation behavior of such surfactants in the molecular level, have been analyzed on the basis of the Hayter and Penfold model for the macro ion solution to compute the interparticle structure factor S(Q) taking into account the screened Coulomb interactions between the dimeric surfactant micelles. The characteristic changes in the SANS spectra of the dimeric surfactant with n = 4 due to variation of temperature have also been investigated. These data are then compared with the SANS characterization data of the corresponding gemini micelles containing tetrahedral ammonium ion based polar headgroups. The critical micellar concentration of each surfactant micelle (cmc) has been determined using pyrene as an extrinsic fluorescence probe. The variation of cmc as a function of spacer chain length has been explained in terms of conformational variation and progressive looping of the spacer into the micellar interior upon increasing the n values. Small-angle neutron-scattering (SANS) cross sections from different mixed micelles composed of surfactants with ammonium headgroups, 16-A(0), 16-Am-n-Am-16], 2Br(-) (where n = 4), 16-I-0, and 16-Im-n-Im-16], 2Br(-) (where n = 4), in aqueous media (D2O) have also been analyzed. The aggregate composition matches with that predicted from the ideal mixing model.

Polymorphs and hydrates of Etoricoxib, a selective COX-2 inhibitor

The crystal structures of two polymorphs and two polymorphic hemihydrates of Etoricoxib are reported. Etoricoxib is a non-steroidal anti-inflammatory drug (NSAID) that is a selective inhibitor of COX-2. It is used in the treatment of various types of inflammation, pain and fever. Clas et al. have reported four polymorphs (labeled I through IV) and two solvates (hemi-and sesquihydrate) of the API in US patent 6,441,002 (Clas et al, US patent 6,441,002, 2002). However, no crystal structures have been reported for any of these forms. A comparison was made between the PXRD patterns reported in patent `002 and the powder spectra simulated from single crystal data. The two polymorphs characterized here correspond to form I and form IV of the patent. Form II of the patent could not be obtained by us with a variety of experimental conditions. Form III of the patent corresponds to hemihydrate II of this study. Form III is therefore not a polymorph of form I and form IV. What we have termed hemihydrate I in this study is obtained under a wide variety of conditions and it is also the only hemihydrate reported as such in the patent. Because the Etoricoxib molecule contains no conventional hydrogen bond donors, there cannot be any strong hydrogen bonds in the crystal structures of forms I and IV. The packing is accordingly characterized by weak hydrogen bonds of the C-H center dot center dot center dot O=S and C-H center dot center dot center dot N type. Thermal data were collected for form I, form IV and hemihydrate I to shed some light on relative stabilities. PXRD diffractograms show the transformation of form IV to form I at elevated temperature, indicating that form I is more stable than form IV. However, this transformation occurs only in samples of form IV that contain some form I; it does not occur in pure form IV. The formation of the two hemihydrates could follow from the known tendency of an acceptor-rich molecule to crystallize as a hydrate.

Influence of Side-Chain on Structural Order and Photophysical Properties in Thiophene Based Diketopyrrolopyrroles: A Systematic Study

In this work, we have synthesized a series of TDPP derivatives with different alkyl groups such as n-hexyl (-C6H13) 3a, 2-ethylhexyl (-(2-C2H5)C6H12) 3b, triethylene glycol mono methyl ether (-(CH2CH2O)(3c)H-3, TEG) 3c, and octadodecyl (-(8-C8H17)C12H22) 3d. N,N dialkylation of Othiophene-diketopyrrolopyrrole (TDPP, 1) strongly influences its solubility, solid state packing, and structural order. These materials allow us to explicitly study the influence of alkyl chain on solid state packing and photophysical properties. TDPP moiety containing two different alkyl groups 3e (TEG and 2-ethylhexyl) and 3f (TEG and n-hexyl) were synthesized for the first time. The absorption spectra of all derivatives exhibited a red shift in solid state when compared to their solution spectra. The type of alkyl chains leads to change in the optical band gaps in solid state. The fluorescence study reveals that TDPP derivatives have strong pi-pi interaction in the solid state and the extent of bathochromic shift is due to combination of intramolecular interaction and formation of aggregates in solid state. This behavior strongly depends on the nature of alkyl chain. The presence of strong C-H center dot center dot center dot O inter chain interactions and CH-pi interactions in solid state exhibits strong influence on the photophysical properties of TDPP chromophore.

From Molecular to Supramolecular: An Exploration into the Modes of Self-assembly in Conformationally Locked Polycyclitols

This brief account highlights the notable findings of our investigation into the supramolecular chemistry of conformationally locked polycyclitols in the solid state. The study was aimed at analyzing the crystal packing and unraveling the modalities of non-covalent interactions (particularly, intramolecular vis-a-vis intermolecular OH center dot center dot center dot O hydrogen bonds) in polyols. The know-how obtained thereof, was successfully utilized to engineer self-assemblies of designer polycyclitols, having hydrogen bond donors and acceptors fettered onto a trans-decalin scaffold. The results seek to draw particular attention to the intrinsic attribute of this rigid carbocyclic framework to lock functional groups into spatially invariant positions and bring potential intramolecular hydrogen bonding partners into favorable interaction geometry to engender predictability in the self-assembly patterns.

Relay Selection for Geographical Forwarding in Sleep-Wake Cycling Wireless Sensor Networks

Our work is motivated by geographical forwarding of sporadic alarm packets to a base station in a wireless sensor network (WSN), where the nodes are sleep-wake cycling periodically and asynchronously. We seek to develop local forwarding algorithms that can be tuned so as to tradeoff the end-to-end delay against a total cost, such as the hop count or total energy. Our approach is to solve, at each forwarding node enroute to the sink, the local forwarding problem of minimizing one-hop waiting delay subject to a lower bound constraint on a suitable reward offered by the next-hop relay; the constraint serves to tune the tradeoff. The reward metric used for the local problem is based on the end-to-end total cost objective (for instance, when the total cost is hop count, we choose to use the progress toward sink made by a relay as the reward). The forwarding node, to begin with, is uncertain about the number of relays, their wake-up times, and the reward values, but knows the probability distributions of these quantities. At each relay wake-up instant, when a relay reveals its reward value, the forwarding node's problem is to forward the packet or to wait for further relays to wake-up. In terms of the operations research literature, our work can be considered as a variant of the asset selling problem. We formulate our local forwarding problem as a partially observable Markov decision process (POMDP) and obtain inner and outer bounds for the optimal policy. Motivated by the computational complexity involved in the policies derived out of these bounds, we formulate an alternate simplified model, the optimal policy for which is a simple threshold rule. We provide simulation results to compare the performance of the inner and outer bound policies against the simple policy, and also against the optimal policy when the source knows the exact number of relays. Observing the good performance and the ease of implementation of the simple policy, we apply it to our motivating problem, i.e., local geographical routing of sporadic alarm packets in a large WSN. We compare the end-to-end performance (i.e., average total delay and average total cost) obtained by the simple policy, when used for local geographical forwarding, against that obtained by the globally optimal forwarding algorithm proposed by Kim et al. 1].

Crystal Structures and Physicochemical Properties of Four New Lamotrigine Multicomponent Forms

In the present study, four new multicomponent forms of lamotrigine (LTG) with selected carboxylic acids, viz. acetic acid, propionic acid, sorbic acid, and glutaric acid, have been identified. Preliminary solid-state characterization was done by differential scanning calorimetry/thermogravimetric, infrared, and powder X-ray diffraction techniques. X-ray single-crystal structure analysis confirmed the proton transfer, stoichiometry, and the molecular composition, revealing all of these to be a new salt/salt-cocrystal/salt monosolvate monohydrate of LTG. All four compounds exhibited both the aminopyridine dimer of LTG (motif 4) and cation-anion dimers between protonated LTG and the carboxylate anion in their crystal structures. Further, these new crystal forms were subjected to solubility studies in water, powder dissolution studies in 0.1 N HCl, and stability studies under humid conditions in comparison with pure LTG base. The solubility of these compounds in water is significantly enhanced compared with that of pure base, which is attributed to the type of packing motifs present in their crystal structures as well as to the lowering of the pH by the acidic coformers. Solid residues of all forms remaining after solubility and dissolution experiments were also assessed for any transformation in water and acidic medium.

Dual Role of Acetate as a Nucleophile and as an Internal Base in Cycloplatination Reaction of sym-N,N `,N `'-Triarylguanidines

Reaction of cis-Cl2Pt(S(O)Me-2)(2)] with 1 equiv of sym-N,N',N `'-triarylguanidines, ArN=C(NHAr)(2) (sym = symmetrical; Ar = 2-MeC6H4 (LH22-tolyl), 2-(MeO)C6H4 (LH22-anisyl), 4-MeC6H4 (LH24-tolyl), 2,5-Me2C6H3 (LH22,5-xylyl), and 2,6-Me2C6H3 (LH22,6-xylyl)) in toluene under reflux condition for 3 h afforded cis- or trans-Cl2Pt(S(O)Me-2)(ArN=C(NHAr)(2))] (Ar = 2-MeC6H4 (1), 2-(MeO)C6H4 (2), 4-MeC6H4 (3), 2,5-h Me2C6H3 (4), and 2,6-Me2C6H3 (5), respectively) in 83-96% yield. Reaction of cis-Cl2Pt(S(O)Me-2)(2)] with 1 equiv of LH22-tolyl and LH24-tolyl in the presence of 1 equiv of NaOAc in methanol under reflux condition for 3 h afforded acetate-substituted products, cis-(AcO)ClPt(S(O)Me-2)(ArN=C(NHAr)(2))] (Ar = 2-MeC6H4 (6) and 4-MeC6H4 (7)) in 83% and 84% yields, respectively. Reaction of cis-Cl2Pt(S(O)Me-2)(2)] with 1 equiv of LH22-anisyl and LH22-tolyl in the presence of 1 equiv of NaOAc in methanol under reflux condition for 3 and 12 h afforded six-membered C,N] platinacycles, Pt{kappa(2)(C,N)-C6H3R-3(NHC(NHAr)(=NAr))-2}Cl(S(O)Me-2)] (Ar = 2-RC6H4; R = OMe (8) and Me (9)), in 92% and 79% yields, respectively. The new complexes have been characterized by analytical and spectroscopic techniques, and further the molecular structures of 1, 2, 4, 5, 6, and 8 have been determined by single-crystal X-ray diffraction. The platinum atom in 1, 4, and 5 exhibited the trans configuration, while that in 2, 6, and 8 exhibited the cis configuration. Complex 6 is shown to be the precursor for 9, and the former is suggested to transform to the latter possibly via an intramolecular C-H activation followed by elimination of AcOH. The solution behavior of new complexes has been studied by multinuclear NMR (H-1, Pt-195, and C-13) spectroscopy. The new complexes exist exclusively as a single isomer (trans (1 and 5) and cis (6 and 7)), a mixture of cis and trans isomers with the former isomer being predominant in the case of 2 and the latter isomer being predominant in the case of 3. Complex 5 in the trans form revealed the presence of one isomer at 0.007 mM concentration and two isomers in about 1.00:0.12 ratio at 0.154 mM concentration as revealed by H-1 NMR spectroscopy, and this has been ascribed to the restricted Pt-S bond rotation at higher concentration. Platinacycle 8 exists as one isomer, while 9 exists as a mixture of seven isomers in solution. The influence of steric factor, pi-acceptor property of the guanidine, subtle solid-state packing forces upon the configuration of the platinum atom, and the number of isomers in solution have been outlined. Factors that accelerate or slow down the cycloplatination reaction, the role of NaOAc, and a plausible mechanism of this reaction have been discussed.

Nanoindentation in Crystal Engineering: Quantifying Mechanical Properties of Molecular Crystals

Nanoindentation is a technique for measuring the elastic modulus and hardness of small amounts of materials. This method, which has been used extensively for characterizing metallic and inorganic solids, is now being applied to organic and metalorganic crystals, and has also become relevant to the subject of crystal engineering, which is concerned with the design of molecular solids with desired properties and functions. Through nanoindentation it is possible to correlate molecular-level properties such as crystal packing, interaction characteristics, and the inherent anisotropy with micro/macroscopic events such as desolvation, domain coexistence, layer migration, polymorphism, and solid-state reactivity. Recent developments and exciting opportunities in this area are highlighted in this Minireview.

Experiments on thermal response of low aspect ratio packed beds at high Reynolds numbers with varying inflow temperature

An experimental study for transient temperature response of low aspect ratio packed beds at high Reynolds numbers for a free stream with varying inlet temperature is presented. The packed bed is used as a compact heat exchanger along with a solid propellant gas-generator, to generate room temperature gases for use in applications such as control actuation and air bottle pressurization. Packed beds of lengths similar to 200 mm and 300 mm were characterized for packing diameter based Reynolds numbers, Re-d ranging from 0.6 x 10(4) to 8.5 x 10(4). The solid packing used in the bed consisted of circle divide 9.5 mm and circle divide 5 mm steel spheres with suitable arrangements to eliminate flow entrance and exit effects. The ratios of packed bed diameter to packing diameter for 9.5 mm and 5 mm sphere packing were similar to 9.5 and 18 respectively, with the average packed bed porosities around 0.4. Gas temperatures were measured at the entry, exit and at three axial locations along centre-line in the packed beds. The solid packing temperature was measured at three axial locations in the packed bed. An average Nusselt number correlation of the form Nu(d) = 3.91Re(d)(05) for Re-d range of 10(4) is proposed. For engineering applications of packed beds such as pebble bed heaters, thermal storage systems, and compact heat exchangers a simple procedure is suggested for calculating unsteady gas temperature at packed bed exit for packing Biot number Bi-d < 0.1. (C) 2012 Elsevier Inc. All rights reserved.

Microscopic dynamics of nanoparticle monolayers at air-water interface

We present results of surface mechanical and particle tracking measurements of nanoparticles trapped at the air-water interface as a function of their areal density. We monitor both the surface pressure (II) and isothermal compression modulus (epsilon) as well as the dynamics of nanoparticle clusters, using fluorescence confocal microscopy while they are compressed to very high density near the two dimensional close packing density Phi similar to 0.82. We observe non-monotonic variation in both epsilon and the dynamic heterogeneity, characterized by the dynamical susceptibility chi(4) with Phi, in such high density monolayers. We provide insight into the underlying nature of such transitions in close packed high density nanoparticle monolayers in terms of the morphology and flexibility of these soft colloidal particles.. We discuss the significance our results in the context of related studies on two dimensional granular or colloidal systems. (C) 2013 Elsevier Inc. All rights reserved.