Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

Tuning the beta-turn segment in designed peptide beta-hairpins: Construction of a stable type I ' beta-turn nucleus and hairpin-helix transition promoting segments

Designed octapeptides Boc-Leu-Val-Val-Aib-(D)Xxx-Leu- Val-Val-OMe ((D)Xxx = (D)Ala, 3a; (D)Val, 3c and (D)Pro, 5a) and Boc-Leu-Phe-Val-Aib-DAla-Leu-Phe-Val-OMe (3b) have been investigated to construct models of a stable type I' beta-turn nucleated hairpin and to generate systems for investigating helix-hairpin conformational transitions. Peptide 5a, which contains a central Aib-(D)Pro segment, is shown to adopt a stable type I' beta-turn nucleated hairpin structure, stabilized by four cross-strand hydrogen bonds. The stability of the structure in diverse solvents is established by the observation of all diagnostic NOEs expected in a beta-hairpin conformation. Replacement of (D)Pro5 by (D)Ala/(D)Val (3a-c) results in sequences that form beta-hairpins in hydrogen bonding solvents like CD3OH and DMSO-d(6). However, in CDCl3 evidence for population of helical conformations is obtained. Peptide 6b (Boc-Leu-Phe-Val-Aib-Aib-Leu-Phe-Val-OMe), which contains a centrally positioned Aib-Aib segment, provides a clear example of a system, which exhibits a helical conformation in CDCl3 and a significant population of both helices and hairpins in CD3OH and DMSO-d(6). The coexistence of multiple conformations is established by the simultaneous observation of diagnostic NOEs. Control over stereochemistry of the central beta-turn permits generation of models for robust beta-hairpins and also for the construction of systems that may be used to probe helix-hairpin conformational transitions. (c) 2006 Wiley Periodicals, Inc.

Dimensional Reduction Near the Deconfinement Transition

When ordinary nuclear matter is heated to a high temperature of ~ 10^12 K, it undergoes a deconfinement transition to a new phase, strongly interacting quark-gluon plasma. While the color charged fundamental constituents of the nuclei, the quarks and gluons, are at low temperatures permanently confined inside color neutral hadrons, in the plasma the color degrees of freedom become dominant over nuclear, rather than merely nucleonic, volumes. Quantum Chromodynamics (QCD) is the accepted theory of the strong interactions, and confines quarks and gluons inside hadrons. The theory was formulated in early seventies, but deriving first principles predictions from it still remains a challenge, and novel methods of studying it are needed. One such method is dimensional reduction, in which the high temperature dynamics of static observables of the full four-dimensional theory are described using a simpler three-dimensional effective theory, having only the static modes of the various fields as its degrees of freedom. A perturbatively constructed effective theory is known to provide a good description of the plasma at high temperatures, where asymptotic freedom makes the gauge coupling small. In addition to this, numerical lattice simulations have, however, shown that the perturbatively constructed theory gives a surprisingly good description of the plasma all the way down to temperatures a few times the transition temperature. Near the critical temperature, the effective theory, however, ceases to give a valid description of the physics, since it fails to respect the approximate center symmetry of the full theory. The symmetry plays a key role in the dynamics near the phase transition, and thus one expects that the regime of validity of the dimensionally reduced theories can be significantly extended towards the deconfinement transition by incorporating the center symmetry in them. In the introductory part of the thesis, the status of dimensionally reduced effective theories of high temperature QCD is reviewed, placing emphasis on the phase structure of the theories. In the first research paper included in the thesis, the non-perturbative input required in computing the g^6 term in the weak coupling expansion of the pressure of QCD is computed in the effective theory framework at an arbitrary number of colors. The two last papers on the other hand focus on the construction of the center-symmetric effective theories, and subsequently the first non-perturbative studies of these theories are presented. Non-perturbative lattice simulations of a center-symmetric effective theory for SU(2) Yang-Mills theory show --- in sharp contrast to the perturbative setup --- that the effective theory accommodates a phase transition in the correct universality class of the full theory. This transition is seen to take place at a value of the effective theory coupling constant that is consistent with the full theory coupling at the critical temperature.

Microstructural evolution during laser resolidification of Fe-25 atom percent Ge alloy

The microstructural evolution of concentrated alloys is relatively less understood both in terms of experiments as well as theory. Laser resolidification represents a powerful technique to study the solidification behavior under controlled growth conditions. This technique has been utilized in the current study to probe experimentally microstructural selection during rapid solidification of concentrated Fe-25 atom pct Ge alloy. Under the equilibrium solidification condition, the alloy undergoes a peritectic reaction between ordered alpha(2) (B2) and its liquid, leading to the formation of ordered hexagonal intermetallic phase epsilon (DO19). In general, the as-cast microstructure consists of epsilon phase and e-p eutectic and alpha(2) that forms as a result of an incomplete peritectic reaction. With increasing laser scanning velocity, the solidification front undergoes a number of morphological transitions leading to the selection of the microstructure corresponding to metastable alpha(2)/beta eutectic to alpha(2) dendrite + alpha(2)/beta eutectic to alpha(2) dendrite. The transition velocities as obtained from the experiments are well characterized. The microstructural selection is discussed using competitive growth kinetics.

Dynamo onset as a first-order transition: Lessons from a shell model for magnetohydrodynamics

We carry out systematic and high-resolution studies of dynamo action in a shell model for magnetohydro-dynamic (MHD) turbulence over wide ranges of the magnetic Prandtl number Pr-M and the magnetic Reynolds number Re-M. Our study suggests that it is natural to think of dynamo onset as a nonequilibrium first-order phase transition between two different turbulent, but statistically steady, states. The ratio of the magnetic and kinetic energies is a convenient order parameter for this transition. By using this order parameter, we obtain the stability diagram (or nonequilibrium phase diagram) for dynamo formation in our MHD shell model in the (Pr-M(-1), Re-M) plane. The dynamo boundary, which separates dynamo and no-dynamo regions, appears to have a fractal character. We obtain a hysteretic behavior of the order parameter across this boundary and suggestions of nucleation-type phenomena.

A model based factorization approach for dense 3D recovery from monocular video

Feature track matrix factorization based methods have been attractive solutions to the Structure-front-motion (Sfnl) problem. Group motion of the feature points is analyzed to get the 3D information. It is well known that the factorization formulations give rise to rank deficient system of equations. Even when enough constraints exist, the extracted models are sparse due the unavailability of pixel level tracks. Pixel level tracking of 3D surfaces is a difficult problem, particularly when the surface has very little texture as in a human face. Only sparsely located feature points can be tracked and tracking error arc inevitable along rotating lose texture surfaces. However, the 3D models of an object class lie in a subspace of the set of all possible 3D models. We propose a novel solution to the Structure-from-motion problem which utilizes the high-resolution 3D obtained from range scanner to compute a basis for this desired subspace. Adding subspace constraints during factorization also facilitates removal of tracking noise which causes distortions outside the subspace. We demonstrate the effectiveness of our formulation by extracting dense 3D structure of a human face and comparing it with a well known Structure-front-motion algorithm due to Brand.

A novel fibre-coated sorbent for rapid removal of heavy metals from wastewater. Sorption of Zinc(II) from dilute aqueous solutions in the presence of high concentrations of common salts

The commercial acrylic fibre "Cashmilon" was partially hydrolyzed to convert a fraction of its nitrile (-CN) groups to carboxylic acid (-COOH) groups and then coated with polyethylenimine (PEI) resin and cross-linked with glutaraldehyde to produce a novel gel-coated fibrous sorbent with multiple functionalities of cationic, anionic and chelating types, and significantly faster sorption kinetics than bead-form sorbents. The sorption properties of the fibrous sorbent were measured using Zn(II) in aqueous solution as the sorbate to determine the effects of pH and the presence of common ions in the solution on the sorption capacity. The rate of sorption on the gel-coated fibre was measured in comparison with that on Amberlite IRA-68 weak-base resin beads, to demonstrate the marked difference between fibre and bead-form sorbents in their kinetic behaviour.

Monoclinic polymorph of Boc-Trp-Ile-Ala-Aib-Ile-Val-Aib-Leu-Aib-Pro-OMe(anhydrous). Parallel packing of 3(10)-/alpha-helices and a transition of helix type

The structures of two crystal forms of Boc-Trp-Ile-Ala-Aib-Ile-Val-Aib-Leu-Aib-Pro-OMe have been determined. The triclinic form (P1, Z = 1) from DMSO/H2O crystallizes as a dihydrate (Karle, Sukumar & Balaram (1986) Proc, Natl, Acad. Sci. USA 83, 9284-9288). The monoclinic form (P2(1), Z = 2) crystallized from dioxane is anhydrous. The conformation of the peptide is essentially the same in both crystal system, but small changes in conformational angles are associated with a shift of the helix from a predominantly alpha-type to a predominantly 3(10)-type. The r.m.s. deviation of 33 atoms in the backbone and C beta positions of residues 2-8 is only 0.29 A between molecules in the two polymorphs. In both space groups, the helical molecules pack in a parallel fashion, rather than antiparallel. The only intermolecular hydrogen bonding is head-to-tail between helices. There are no lateral hydrogen bonds. In the P2(1) cell, a = 9.422(2) A, b = 36.392(11) A, c = 10.548(2) A, beta = 111.31(2) degrees and V = 3369.3 A for 2 molecules of C60H97N11O13 per cell.

Scanning Tunneling Spectroscopy on Pr0.68Pb0.32MnO3 single crystals

Scanning tunneling microscopy/spectroscopy studies were carried out on single crystals of colossal magnetoresistive manganite Pr0. 68Pb0.32MnO3 at different temperatures in order to probe their spatial homogeneity across the metal-insulator transition temperature TM-I(similar to 255 K). A metallic behavior of the local conductance was observed for temperatures T < TM-I. Zero bias conductance (dI/dV)v=(0), which is directly proportional to the local surface density of states at the Fermi level, shows a single distribution at temperatures T < 200 K suggesting a homogeneous electronic phase at low temperatures. In a narrow temperature window of 200 K < T < TM-I, however, an inhomogeneous distribution of (dI/dV)v=(0) has been observed. This result gives evidence for phase separation in the transition region in this compound.