An Actor-Critic Algorithm for Finite Horizon Markov Decision Processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Nature of Electronic States in Atomically Thin MoS(2) Field-Effect Transistors

We present low-temperature electrical transport experiments in five field-effect transistor devices consisting of monolayer, bilayer, and trilayer MoS(2) films, mechanically exfoliated onto Si/SiO(2) substrate. Our experiments reveal that the electronic states In all films are localized well up to room temperature over the experimentally accessible range of gate voltage. This manifests in two-dimensional (2D) variable range hopping (VRH) at high temperatures, while below similar to 30 K, the conductivity displays oscillatory structures In gate voltage arising from resonant tunneling at the localized sites. From the correlation energy (T(0)) of VRH and gate voltage dependence of conductivity, we suggest that Coulomb potential from trapped charges In the substrate is the dominant source of disorder in MoS(2) field-effect devices, which leads to carrier localization, as well.

Structure and dynamics of protein excited states with millisecond lifetimes

An in-depth understanding of biological processes often requires detailed atomic resolution structures of the molecules involved. However in solution where most of these processes occur the conformation of biomolecules like RNA, DNA and proteins is not static but fluctuates. Routinely used structural techniques like X-ray crystallography, NMR spectroscopy and cryo-electron microscopy have almost always been used to determine the structure of the dominant conformation or obtain an average structure of the biomolecule in solution with very little detailed information regarding the dynamics of these molecules in solution. Over the last few years, NMR based methods have been developed to study the dynamics of these biomolecules in solution in a site-specific manner with the aim of generating structures of the different conformations that these molecules can adopt in solution. One powerful technique is the Carr-Purcell-Meiboom-Gill (CPMG) relaxation dispersion experiment, which can be used to detect and characterize protein excited states that are populated for as less as 0.5% of the time with ∼0.5–10 millisecond lifetimes. Due to recent advances in NMR pulse sequences and labeling methodology, it is now possible to determine the structures of these transiently populated excited states with millisecond lifetimes by obtaining accurate chemical shifts, residual dipolar couplings (RDCs) and residual chemical shift anisotropies (RCSAs) of these excited states. In these excited states the dynamics of some methyl containing residues can also be studied.

XPS and x-ray absorption-edge studies of the surface and bulk valence states of cerium in ceco2

XPS and LIII X-ray absorption edge studies regarding the valence state of cerium have been carried out on the intermetallic compounds CeCo2, which becomes superconducting at low temperatures. It is observed from XPS that the surface shows both Ce3+ and Ce4+ valence states, while the X-ray absorption edge studies reveal only Ce4+ in the bulk. Thus valence fluctuation and superconductivity do not coexist in the bulk of this compound.

Dynamic properties of sand from dry to fully saturated states

By using bender and extender elements test, the velocities of the primary and shear waves, V(P) and V(s) respectively, were measured for a sandy material by gradually varying the degree of saturation, S(r), between the dry and fully saturated states. The effect on the results of varying the relative density and effective confining pressure was also studied. The measurements clearly reveal that for a certain optimum S(r), which is around 0.7-0.9% for the chosen sand, the value of the shear modulus G reaches a maximum value, whereas the corresponding Poisson's ratio nu attains a minimum value. The values of the shear modulus corresponding to S(r) approximate to 0% and S(r) = 100% tend towards the same value. For values of Skempton's B parameter greater than 0.99, the values of V(P) and nu rise very sharply to those of water. The predictions from Biot's theory with respect to the variation of V(P) with S(r) match well with the measured experimental data.

Charge transfer complex states in diketopyrrolopyrrole polymers and fullerene blends: Implications for organic solar cell efficiency

The spectral photocurrent characteristics of two donor-acceptor diketopyrrolopyrrole (DPP)-based copolymers (PDPP-BBT and TDPP-BBT) blended with a fullerene derivative [6,6]-phenyl C-61-butyric acid methyl ester (PCBM) were studied using Fourier-transform photocurrent spectroscopy (FTPS) and monochromatic photocurrent (PC) method. PDPP-BBT: PCBM shows the onset of the lowest charge transfer complex (CTC) state at 1.42 eV, whereas TDPP-BBT: PCBM shows no evidence of the formation of a midgap CTC state. The FTPS and PC spectra of P3HT:PCBM are also compared. The larger singlet state energy difference of TDPP-BBT and PCBM compared to PDPP-BBT/P3HT and PCBM obliterates the formation of a midgap CTC state resulting in an enhanced photovoltaic efficiency over PDPP-BBT: PCBM. (C) 2011 American Institute of Physics. [doi:10.1063/1.3670043]

Symmetrized DMRG studies of low-energy electronic states of poly-para-phenylene (PPP) and poly-para-phenylenevinylene (PPV) systems

Symmetrized DMRG calculations on long oligomers of poly- para-phenylene (PPP) and poly-para-phenylene vinylene (PPV) systems within a `U-V' model have been carried out to obtain the one-photon, two-photon and singlet-triplet gaps in these systems. The extrapolated gaps (in eV) are 2.89, 3.76 and 2.72 in PPP and 3.01, 3.61 and 2.23 in PPV for the one- photon, two-photon and spin gaps respectively. By studying doped systems, we have obtained the exciton binding energies. The larger exciton binding energies, compared to strongly dimerized linear chains emphasizes the role of topology in these polymers. Bond orders, charge and spin correlations in the low-lying states bring out the similarities between the lowest one-photon, the lowest triplet and the lowest bipolaronic states in these systems. The two-photon state bond orders show evidence for strong localization of this excitation in both PPP and PPV systems.

Performance analysis of UDP with energy efficient link layer on Markov fading channels

In this paper, we analyze the throughput and energy efficiency performance of user datagram protocol (UDP) using linear, binary exponential, and geometric backoff algorithms at the link layer (LL) on point-to-point wireless fading links. Using a first-order Markov chain representation of the packet success/failure process on fading channels, we derive analytical expressions for throughput and energy efficiency of UDP/LL with and without LL backoff. The analytical results are verified through simulations. We also evaluate the mean delay and delay variation of voice packets and energy efficiency performance over a wireless link that uses UDP for transport of voice packets and the proposed backoff algorithms at the LL. We show that the proposed LL backoff algorithms achieve energy efficiency improvement of the order of 2-3 dB compared to LL with no backoff, without compromising much on the throughput and delay performance at the UDP layer. Such energy savings through protocol means will improve the battery life in wireless mobile terminals.

Statistical properties of entropy-consuming fluctuations in jammed states of laponite suspensions: Fluctuation relations and generalized Gumbel distribution

We report a universal large deviation behavior of spatially averaged global injected power just before the rejuvenation of the jammed state formed by an aging suspension of laponite clay under an applied stress. The probability distribution function (PDF) of these entropy consuming strongly non-Gaussian fluctuations follow an universal large deviation functional form described by the generalized Gumbel (GG) distribution like many other equilibrium and nonequilibrium systems with high degree of correlations but do not obey the Gallavotti-Cohen steady-state fluctuation relation (SSFR). However, far from the unjamming transition (for smaller applied stresses) SSFR is satisfied for both Gaussian as well as non-Gaussian PDF. The observed slow variation of the mean shear rate with system size supports a recent theoretical prediction for observing GG distribution.

Delay Optimal Event Detection on Ad Hoc Wireless Sensor Networks

We consider a small extent sensor network for event detection, in which nodes periodically take samples and then contend over a random access network to transmit their measurement packets to the fusion center. We consider two procedures at the fusion center for processing the measurements. The Bayesian setting, is assumed, that is, the fusion center has a prior distribution on the change time. In the first procedure, the decision algorithm at the fusion center is network-oblivious and makes a decision only when a complete vector of measurements taken at a sampling instant is available. In the second procedure, the decision algorithm at the fusion center is network-aware and processes measurements as they arrive, but in a time-causal order. In this case, the decision statistic depends on the network delays, whereas in the network-oblivious case, the decision statistic does not. This yields a Bayesian change-detection problem with a trade-off between the random network delay and the decision delay that is, a higher sampling rate reduces the decision delay but increases the random access delay. Under periodic sampling, in the network-oblivious case, the structure of the optimal stopping rule is the same as that without the network, and the optimal change detection delay decouples into the network delay and the optimal decision delay without the network. In the network-aware case, the optimal stopping problem is analyzed as a partially observable Markov decision process, in which the states of the queues and delays in the network need to be maintained. A sufficient decision statistic is the network state and the posterior probability of change having occurred, given the measurements received and the state of the network. The optimal regimes are studied using simulation.

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Hilbert W*-modules and coherent states

Hilbert C*-module valued coherent states was introduced earlier by Ali, Bhattacharyya and Shyam Roy. We consider the case when the underlying C*-algebra is a W*-algebra. The construction is similar with a substantial gain. The associated reproducing kernel is now algebra valued, rather than taking values in the space of bounded linear operators between two C*-algebras.

Analytical Field-Effect Method for Extraction of Subgap States in Thin-Film Transistors

We present an analytical field-effect method to extract the density of subgap states (subgap DOS) in amorphous semiconductor thin-film transistors (TFTs), using a closed-form relationship between surface potential and gate voltage. By accounting the interface states in the subthreshold characteristics, the subgap DOS is retrieved, leading to a reasonably accurate description of field-effect mobility and its gate voltage dependence. The method proposed here is very useful not only in extracting device performance but also in physically based compact TFT modeling for circuit simulation.

Continuum theory of edge states of topological insulators: variational principle and boundary conditions

We develop a continuum theory to model low energy excitations of a generic four-band time reversal invariant electronic system with boundaries. We propose a variational energy functional for the wavefunctions which allows us to derive natural boundary conditions valid for such systems. Our formulation is particularly suited for developing a continuum theory of the protected edge/surface excitations of topological insulators both in two and three dimensions. By a detailed comparison of our analytical formulation with tight binding calculations of ribbons of topological insulators modelled by the Bernevig-Hughes-Zhang (BHZ) Hamiltonian, we show that the continuum theory with a natural boundary condition provides an appropriate description of the low energy physics.