Solving MDPs using two-timescale simulated annealing with multiplicative weights

We develop extensions of the Simulated Annealing with Multiplicative Weights (SAMW) algorithm that proposed a method of solution of Finite-Horizon Markov Decision Processes (FH-MDPs). The extensions developed are in three directions: a) Use of the dynamic programming principle in the policy update step of SAMW b) A two-timescale actor-critic algorithm that uses simulated transitions alone, and c) Extending the algorithm to the infinite-horizon discounted-reward scenario. In particular, a) reduces the storage required from exponential to linear in the number of actions per stage-state pair. On the faster timescale, a 'critic' recursion performs policy evaluation while on the slower timescale an 'actor' recursion performs policy improvement using SAMW. We give a proof outlining convergence w.p. 1 and show experimental results on two settings: semiconductor fabrication and flow control in communication networks.

Thermodynamic analysis of three state denaturation of Peanut Agglutinin

Peanut Agglutinin (PNA) is a homotetrameric protein with a very unusual open quaternary structure. During denaturation, it first dissociates into a molten globule like state, which subsequently undergoes complete denaturation. Urea denaturation of PNA at neutral pH has been studied by intrinsic fluorescence spectroscopy and has been fitted to a three state model, A(4) double left right arrow 4I double left right arrow 4U, to get all the relevant thermodynamic parameters. Urea denaturation leads to continuous red shift of wavelength maxima. The molten globule like state is formed in a short range of urea concentration. Refolding of the denatured PNA has been attempted by intrinsic fluorescence study. Refolding by instantaneous dilution shows the occurrence of the formation of an intermediate at a relatively rapid rate, within few seconds. The transition from PNA tetramer to molten globule like state is found to have a Delta G value of similar to 33 kcal/mole while it is similar to 8 kcal/mole for the transition from molten globule like state to a completely denatured state. This in turn indicates that the tetramerization in PNA contributes significantly to the stability of the oligomer.

Molten globule-like state of peanut lectin monomer retains its carbohydrate specificity - Implications in protein folding and legume lectin oligomerization

A central question in biological chemistry is the minimal structural requirement of a protein that would determine its specificity and activity, the underlying basis being the importance of the entire structural element of a protein with regards to its activity vis a vis the overall integrity and stability of the protein. Although there are many reports on the characterization of protein folding/ unfolding intermediates, with considerable secondary structural elements but substantial loss of tertiary structure, none of them have been reported to show any activity toward their respective ligands. This may be a result of the conditions under which such intermediates have been isolated or due to the importance of specific structural elements for the activity. In this paper we report such an intermediate in the unfolding of peanut agglutinin that seems to retain, to a considerable degree, its carbohydrate binding specificity and activity. This result has significant implications on the molten globule state during the folding pathway(s) of proteins in general and the quaternary association in legume lectins in particular, where precise subunit topology is required for their biologic activities.

The dynamics of solvation of an electron in the image potential state by a layer of polar adsorbates

Recently, ultrafast two-photon photoemission has been used to study electron solvation at a two-dimensional metal/polar adsorbate interfaces [A. Miller , Science 297, 1163 (2002)]. The electron is bound to the surface by the image interaction. Earlier we have suggested a theoretical description of the states of the electron interacting with a two-dimensional layer of the polar adsorbate [K. L. Sebastian , J. Chem. Phys. 119, 10350 (2003)]. In this paper we have analyzed the dynamics of electron solvation, assuming a trial wave function for the electron and the solvent polarization and then using the Dirac-Frenkel variational method to determine it. The electron is initially photoexcited to a delocalized state, which has a finite but large size, and causes the polar molecules to reorient. This reorientation acts back on the electron and causes its wave function to shrink, which will cause further reorientation of the polar molecules, and the process continues until the electron gets self-trapped. For reasonable values for the parameters, we are able to obtain fair agreement with the experimental observations. (c) 2005 American Institute of Physics.

Solution of MDPS using simulation-based value iteration

This article proposes a three-timescale simulation based algorithm for solution of infinite horizon Markov Decision Processes (MDPs). We assume a finite state space and discounted cost criterion and adopt the value iteration approach. An approximation of the Dynamic Programming operator T is applied to the value function iterates. This 'approximate' operator is implemented using three timescales, the slowest of which updates the value function iterates. On the middle timescale we perform a gradient search over the feasible action set of each state using Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates, thus finding the minimizing action in T. On the fastest timescale, the 'critic' estimates, over which the gradient search is performed, are obtained. A sketch of convergence explaining the dynamics of the algorithm using associated ODEs is also presented. Numerical experiments on rate based flow control on a bottleneck node using a continuous-time queueing model are performed using the proposed algorithm. The results obtained are verified against classical value iteration where the feasible set is suitably discretized. Over such a discretized setting, a variant of the algorithm of [12] is compared and the proposed algorithm is found to converge faster.

Effect of boundary condition description on convergence of solution in a boundary-value problem

It is well known that the numerical accuracy of a series solution to a boundary-value problem by the direct method depends on the technique of approximate satisfaction of the boundary conditions and on the stage of truncation of the series. On the other hand, it does not appear to be generally recognized that, when the boundary conditions can be described in alternative equivalent forms, the convergence of the solution is significantly affected by the actual form in which they are stated. The importance of the last aspect is studied for three different techniques of computing the deflections of simply supported regular polygonal plates under uniform pressure. It is also shown that it is sometimes possible to modify the technique of analysis to make the accuracy independent of the description of the boundary conditions.

2(n)-SEMA-a robust solid state nuclear magnetic resonance experiment for measuring heteronuclear dipolar couplings in static oriented systems using effective transverse spin-lock

Separated local field (SLF) spectroscopy is a powerful technique to measure heteronuclear dipolar couplings. The method provides site-specific dipolar couplings for oriented samples such as membrane proteins oriented in lipid bilayers and liquid crystals. A majority of the SLF techniques utilize the well-known Polarization Inversion Spin Exchange at Magic Angle (PISEMA) pulse scheme which employs spin exchange at the magic angle under Hartmann-Hahn match. Though PISEMA provides a relatively large scaling factor for the heteronuclear dipolar coupling and a better resolution along the dipolar dimension, it has a few shortcomings. One of the major problems with PISEMA is that the sequence is very much sensitive to proton carrier offset and the measured dipolar coupling changes dramatically with the change in the carrier frequency. The study presented here focuses on modified PISEMA sequences which are relatively insensitive to proton offsets over a large range. In the proposed sequences, the proton magnetization is cycled through two quadrants while the effective field is cycled through either two or four quadrants. The modified sequences have been named as 2(n)-SEMA where n represents the number of quadrants the effective field is cycled through. Experiments carried out on a liquid crystal and a single crystal of a model peptide demonstrate the usefulness of the modified sequences. A systematic study under various offsets and Hartmann-Hahn mismatch conditions has been carried out and the performance is compared with PISEMA under similar conditions.

Benedict-Webb-Rubin equation of state constants for N2O4 ⇄ 2NO2,NO,and O2

Benedict-Webb-Rubin equation of state constants for NO, O2, and the equilibrium mixture N2O4 ⇄ 2NO2 are reported.

A class of one-dimensional steady-state flows in radiation-gas-dynamics

The study of steady-state flows in radiation-gas-dynamics, when radiation pressure is negligible in comparison with gas pressure, can be reduced to the study of a single first-order ordinary differential equation in particle velocity and radiation pressure. The class of steady flows, determined by the fact that the velocities in two uniform states are real, i.e. the Rankine-Hugoniot points are real, has been discussed in detail in a previous paper by one of us, when the Mach number M of the flow in one of the uniform states (at x=+∞) is greater than one and the flow direction is in the negative direction of the x-axis. In this paper we have discussed the case when M is less than or equal to one and the flow direction is still in the negative direction of the x-axis. We have drawn the various phase planes and the integral curves in each phase plane give various steady flows. We have also discussed the appearance of discontinuities in these flows.

Convergence correction in colloid scattering

The error introduced in depolarisation measurements due to the convergence of the incident beam has been investigated theoretically as well as experimentally for the case of colloid scattering, where the particles are not small compared to the wavelength of light. Assuming the scattering particles to be anisotropic rods, it is shown that, when the incident unpolarised light is condensed by means of a lens with a circular aperture, the observed depolarisation ratio ϱ u is given by ϱ u = ϱ u0 + 5/3 θ2 where ϱ u0 is the true depolarisation for incident parallel light, and θ the semi-angle of convergence. Appropriate formulae are derived when the incident beam is polarised vertically and horizontally. Experiments performed on six typical colloids support the theoretical conclusions. Other immediate consequences of the theory are also discussed.

Excited-state structures of molecules with nearby electronic states

Vibronic coupling among the nearby excited electronic states via the in-plane and the out-of-plane nuclear motions is examined in benzene, pyrazine, formaldehyde and thioformaldehyde. Results reveal that in benzene the structure distorts via the most active nuclear bending (planar) motion while in the other molecules the structures distort through an out-of-plane bending motion in their respective lowest excited states.

Holographic optical elements: State-of-the-art review: Part 2

This is the second part of a two part review on the state-of-the-art in holographic optical elements (HOEs). The aspects of fabrication, evaluation, and applications of HOEs, are discussed in this part. It details the direction of future efforts towards finding work-horse type recording media, developing new methods for the evaluation of HOE, and identifying the areas of application where HOEs are to be considered as indispensable components/tools. Finally a summary of all the suggestions for future work made in the two parts is displayed in Table 2 of this part of the review.

Holographic optical elements — State-of-the-art review:Part I

A state-of-the-art review on holographic optical elements (HOE) is presented in two parts. In Part I a conceptual overview and an assessment of the current status on the design of HOE have been included. It is pointed out that HOE development based on the use of squeezed light, speckle, non-linear recording, comparative studies between optics and communication approaches, are some of the promising directions for future research in this vital area of photonics.

Prospects for wind energy utilisation in Karnataka State

An examination of the data available at 22 meteorological stations in Karnataka State shows that wind velocities in the State as a whole are neither spectacularly high nor negligibly low. The highest winds (annual mean of around 13 km/hr) are experienced in parts of the northern maidan region of the State (Gulbarga, Raichur and Bidar districts) and in Bangalore. The winds are strongly seasonal: typically, the five monsoon months May-September account for about 80% of the annual wind energy flux. Although the data available are inadequate to make precise estimates, they indicate that the total wind energy potential of the State is about an order of magnitude higher than the current electrical energy consumption. The possible exploitation of wind energy for applications in rural areas therefore requires serious consideration, but it is argued that to be successful it is essential to formulate an integrated and carefully planned programme. The output of current windpumps needs to be increased; a doubling should be feasible by the design of suitable load-matching devices. The first cost has to be reduced by careful design, by the use of local materials and skills and by employing a labour-intensive technology. A consideration of the agricultural factors in the northern maidan region of the State shows that there is likely to be a strong need for mechanical assistance in supplemental and life-saving irrigation for the dry crops characteristic of the area. A technological target for a windmill that could find applications in this area would be one with a rotor diameter of about 10 m that can lift about 10,000 litres of water per hour in winds of 10 km/hr (2.8 m/s) hourly average speed and costs less than about Rs 10,000. Although no such windmills exist as of today, the authors believe that achievement of this target is feasible. An examination of various possible scenarios for the use of windmills in this area suggests that with a windpump costing about Rs 12,000, a three hectare farm growing two dry crops a year can expect an annual return of about 150% from an initial investment of about Rs 15,000. It is concluded that it should be highly worthwhile to undertake a coordinated programme for wind energy development that will include more detailed wind surveys in the northern maidan area (as well as some others, such as the Western Ghats), the development of suitable windmill designs and a study of their applications to agriculture as well as to other fields.