Markov chain modeling of evolution of strains in reinforced concrete flexural beams

From the analysis of experimentally observed variations in surface strains with loading in reinforced concrete beams, it is noted that there is a need to consider the evolution of strains (with loading) as a stochastic process. Use of Markov Chains for modeling stochastic evolution of strains with loading in reinforced concrete flexural beams is studied in this paper. A simple, yet practically useful, bi-level homogeneous Gaussian Markov Chain (BLHGMC) model is proposed for determining the state of strain in reinforced concrete beams. The BLHGMC model will be useful for predicting behavior/response of reinforced concrete beams leading to more rational design.

Under-representation of intrinsic terminators across bacterial genomic islands: Rho as a principal regulator of xenogenic DNA expression

Two transcription termination mechanisms - intrinsic and Rho-dependent - have evolved in bacteria. The Rho factor occurs in most bacterial lineages, and has been hypothesized to play a global regulatory role. Genome-wide studies using microarray, 2D-gel electrophoresis and ChIP-chip provided evidence that Rho serves to silence transcription from horizontally acquired genes and prophages in Escherichia coli K-12, implicating the factor to be a part of the ``cellular immune mechanism'' protecting against deleterious phages and aberrant gene expression from acquired xenogenic DNA. We have investigated this model by adopting an alternate in silico approach and have extended the study to other species. Our analysis shows that several genomic islands across diverse phyla have under-representation of intrinsic terminators, similar to that experimentally observed in E. coli K-12. This implies that Rho-dependent termination is the predominant process operational in these islands and that silencing of foreign DNA is a conserved function of Rho. From the present analysis, it is evident that horizontally acquired islands have lost intrinsic terminators to facilitate Rho-dependent termination. These results underscore the importance of Rho as a conserved, genome-wide sentinel that regulates potentially toxic xenogenic DNA. (C) 2012 Elsevier B.V. All rights reserved.

Lithium D-Isoascorbate Monohydrate, a New Nonlinear Optical Material

Single crystals of lithium D-isoascorbate monohydrate (LDAM), (C6H7O6Li center dot H2O), are grown by a solution growth method. The crystal structure of LDAM is solved using single crystal X-ray diffraction. The space group is orthorhombic P2(1)2(1)2(1) with four formula units per unit cell and lattice parameters a = 7.7836(3) angstrom, b = 8.7456(3) angstrom, and c = 11.0368(4) angstrom. Solubility of the material in water is determined thermogravimetrically and found to have a positive temperature coefficient of solubility. Large optical quality single crystals are subsequently grown from aqueous solution by a slow cooling method. The crystal has a bulky prismatic habit and among the prominent faces the c face appears as the only principal morphological face. The crystal exhibits a (010) cleavage. Dielectric spectroscopy reveals a nearly Debye type Cole-Cole behavior with anisotropy in relaxation. Optical transmission range is found to be from 300 to 1400 nm. The principal refractive indices of this biaxial crystal, measured using Brewster's angle method, at wavelengths 405, 543, and 632.8 nm, show high dispersion. The crystal is negative biaxial with 2V(z) = 107.8 degrees (405 nm) and belongs to the Hobden class 3. Theoretically generated type 1 and type 2 second order phase matching curves match very well with the experimental results. The second-order nonlinear coefficient d(14) was determined to be 7 x 10(-13) m/V. For the optimum phase matching direction (type 2), the second-order effective nonlinear coefficient and the walk off angle are determined to be 0.84 times d(14) and 3.5 degrees respectively. The crystal possesses high multiple surface damage thresholds of 18 GW/cm(2) and 8 GW/cm(2) at laser wavelengths 1064 and 532 nm, respectively.

Methodology for Pavement Design Reliability and Back Analysis Using Markov Chain Monte Carlo Simulation

Given the increasing cost of designing and building new highway pavements, reliability analysis has become vital to ensure that a given pavement performs as expected in the field. Recognizing the importance of failure analysis to safety, reliability, performance, and economy, back analysis has been employed in various engineering applications to evaluate the inherent uncertainties of the design and analysis. The probabilistic back analysis method formulated on Bayes' theorem and solved using the Markov chain Monte Carlo simulation method with a Metropolis-Hastings algorithm has proved to be highly efficient to address this issue. It is also quite flexible and is applicable to any type of prior information. In this paper, this method has been used to back-analyze the parameters that influence the pavement life and to consider the uncertainty of the mechanistic-empirical pavement design model. The load-induced pavement structural responses (e.g., stresses, strains, and deflections) used to predict the pavement life are estimated using the response surface methodology model developed based on the results of linear elastic analysis. The failure criteria adopted for the analysis were based on the factor of safety (FOS), and the study was carried out for different sample sizes and jumping distributions to estimate the most robust posterior statistics. From the posterior statistics of the case considered, it was observed that after approximately 150 million standard axle load repetitions, the mean values of the pavement properties decrease as expected, with a significant decrease in the values of the elastic moduli of the expected layers. An analysis of the posterior statistics indicated that the parameters that contribute significantly to the pavement failure were the moduli of the base and surface layer, which is consistent with the findings from other studies. After the back analysis, the base modulus parameters show a significant decrease of 15.8% and the surface layer modulus a decrease of 3.12% in the mean value. The usefulness of the back analysis methodology is further highlighted by estimating the design parameters for specified values of the factor of safety. The analysis revealed that for the pavement section considered, a reliability of 89% and 94% can be achieved by adopting FOS values of 1.5 and 2, respectively. The methodology proposed can therefore be effectively used to identify the parameters that are critical to pavement failure in the design of pavements for specified levels of reliability. DOI: 10.1061/(ASCE)TE.1943-5436.0000455. (C) 2013 American Society of Civil Engineers.

Non-Stationary Semi-Markov Decision Processes on a Finite Horizon

We introduce and study a class of non-stationary semi-Markov decision processes on a finite horizon. By constructing an equivalent Markov decision process, we establish the existence of a piecewise open loop relaxed control which is optimal for the finite horizon problem.

Wheeler-Feynman absorber revisited: a useful technique to calculate decay rates and lifetimes in small scale optical systems

The Wheeler-Feynman (WF) absorber theory of radiation though no more of interest in explaining self interaction of an electron, can be very useful in today's research in small scale optical systems. The significance of the WF absorber is the use of time-symmetrical solution of Maxwell's equations as opposed to only the retarded solution. The radiative coupling of emitters to nano wires in the near field and change in their lifetimes due to small mode volume enclosures have been elucidated with the retarded solutions before. These solutions have also been shown to agree with quantum electrodynamics, thus allowing for classical electromagnetic approaches in such problems. It is here assumed that the radiative coupling of the emitter with a body is in proportion to its contribution to the classical force of radiative reaction as derived in the WF absorber theory. Representing such nano structures as a partial WF absorber acting on the emitter makes the computations considerably easier than conventional electromagnetic solutions for full boundary conditions.

A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.

Generalized eigenvalue decomposition of the field autocorrelation in correlation diffusion of photons in turbid media

We propose a novel numerical method based on a generalized eigenvalue decomposition for solving the diffusion equation governing the correlation diffusion of photons in turbid media. Medical imaging modalities such as diffuse correlation tomography and ultrasound-modulated optical tomography have the (elliptic) diffusion equation parameterized by a time variable as the forward model. Hitherto, for the computation of the correlation function, the diffusion equation is solved repeatedly over the time parameter. We show that the use of a certain time-independent generalized eigenfunction basis results in the decoupling of the spatial and time dependence of the correlation function, thus allowing greater computational efficiency in arriving at the forward solution. Besides presenting the mathematical analysis of the generalized eigenvalue problem on the basis of spectral theory, we put forth the numerical results that compare the proposed numerical method with the standard technique for solving the diffusion equation.

Application of the random eigenvalue problem in forced response analysis of a linear stochastic structure

The random eigenvalue problem arises in frequency and mode shape determination for a linear system with uncertainties in structural properties. Among several methods of characterizing this random eigenvalue problem, one computationally fast method that gives good accuracy is a weak formulation using polynomial chaos expansion (PCE). In this method, the eigenvalues and eigenvectors are expanded in PCE, and the residual is minimized by a Galerkin projection. The goals of the current work are (i) to implement this PCE-characterized random eigenvalue problem in the dynamic response calculation under random loading and (ii) to explore the computational advantages and challenges. In the proposed method, the response quantities are also expressed in PCE followed by a Galerkin projection. A numerical comparison with a perturbation method and the Monte Carlo simulation shows that when the loading has a random amplitude but deterministic frequency content, the proposed method gives more accurate results than a first-order perturbation method and a comparable accuracy as the Monte Carlo simulation in a lower computational time. However, as the frequency content of the loading becomes random, or for general random process loadings, the method loses its accuracy and computational efficiency. Issues in implementation, limitations, and further challenges are also addressed.

A Markov Decision Theoretic Approach to Pilot Allocation and Receive Antenna Selection

This paper considers antenna selection (AS) at a receiver equipped with multiple antenna elements but only a single radio frequency chain for packet reception. As information about the channel state is acquired using training symbols (pilots), the receiver makes its AS decisions based on noisy channel estimates. Additional information that can be exploited for AS includes the time-correlation of the wireless channel and the results of the link-layer error checks upon receiving the data packets. In this scenario, the task of the receiver is to sequentially select (a) the pilot symbol allocation, i.e., how to distribute the available pilot symbols among the antenna elements, for channel estimation on each of the receive antennas; and (b) the antenna to be used for data packet reception. The goal is to maximize the expected throughput, based on the past history of allocation and selection decisions, and the corresponding noisy channel estimates and error check results. Since the channel state is only partially observed through the noisy pilots and the error checks, the joint problem of pilot allocation and AS is modeled as a partially observed Markov decision process (POMDP). The solution to the POMDP yields the policy that maximizes the long-term expected throughput. Using the Finite State Markov Chain (FSMC) model for the wireless channel, the performance of the POMDP solution is compared with that of other existing schemes, and it is illustrated through numerical evaluation that the POMDP solution significantly outperforms them.

On the unramified principal series of GL(3) over non-archimedean local fields

Let F be a non-archimedean local field and let O be its ring of integers. We give a complete description of the irreducible constituents of the restriction of the unramified principal series representations of GL(3)(F) to GL(3)(O). (C) 2013 Elsevier Inc. All rights reserved.

Third order trace formula

In J. Funct. Anal. 257 (2009) 1092-1132, Dykema and Skripka showed the existence of higher order spectral shift functions when the unperturbed self-adjoint operator is bounded and the perturbation is Hilbert-Schmidt. In this article, we give a different proof for the existence of spectral shift function for the third order when the unperturbed operator is self-adjoint (bounded or unbounded, but bounded below).

Zero-sum risk-sensitive stochastic games on a countable state space

Infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled Markov chains with countably many states are analyzed. Upper and lower values for these games are established. The existence of value and saddle-point equilibria in the class of Markov strategies is proved for the discounted-cost game. The existence of value and saddle-point equilibria in the class of stationary strategies is proved under the uniform ergodicity condition for the ergodic-cost game. The value of the ergodic-cost game happens to be the product of the inverse of the risk-sensitivity factor and the logarithm of the common Perron-Frobenius eigenvalue of the associated controlled nonlinear kernels. (C) 2013 Elsevier B.V. All rights reserved.

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

Asynchronous Gossip for Averaging and Spectral Ranking

We consider two variants of the classical gossip algorithm. The first variant is a version of asynchronous stochastic approximation. We highlight a fundamental difficulty associated with the classical asynchronous gossip scheme, viz., that it may not converge to a desired average, and suggest an alternative scheme based on reinforcement learning that has guaranteed convergence to the desired average. We then discuss a potential application to a wireless network setting with simultaneous link activation constraints. The second variant is a gossip algorithm for distributed computation of the Perron-Frobenius eigenvector of a nonnegative matrix. While the first variant draws upon a reinforcement learning algorithm for an average cost controlled Markov decision problem, the second variant draws upon a reinforcement learning algorithm for risk-sensitive control. We then discuss potential applications of the second variant to ranking schemes, reputation networks, and principal component analysis.