Pricing Defaultable Bonds in a Markov Modulated Market

We address the problem of pricing defaultable bonds in a Markov modulated market. Using Merton's structural approach we show that various types of defaultable bonds are combination of European type contingent claims. Thus pricing a defaultable bond is tantamount to pricing a contingent claim in a Markov modulated market. Since the market is incomplete, we use the method of quadratic hedging and minimal martingale measure to derive locally risk minimizing derivative prices, hedging strategies and the corresponding residual risks. The price of defaultable bonds are obtained as solutions to a system of PDEs with weak coupling subject to appropriate terminal and boundary conditions. We solve the system of PDEs numerically and carry out a numerical investigation for the defaultable bond prices. We compare their credit spreads with some of the existing models. We observe higher spreads in the Markov modulated market. We show how business cycles can be easily incorporated in the proposed framework. We demonstrate the impact on spreads of the inclusion of rare states that attempt to capture a tight liquidity situation. These states are characterized by low risk-free interest rate, high payout rate and high volatility.

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Synthesis and Characterization of a Fluorescent Analogue of Cyclic di-GMP

Cyclic di-GMP (c-di-GMP), a ubiquitous bacterial second messenger, has emerged as a key controller of several biological processes. Numbers of reports that deal with the mechanistic aspects of this second messenger have appeared in the literature. However, the lack of a reporter tag attached to the c-di-GMP at times limits the understanding of further details. In this study, we have chemically coupled N-methylisatoic anhydride (MANT) with c-di-GMP, giving rise to Mant-(c-di-GMP) or MANT-CDG. We have characterized the chemical and physical properties and spectral behavior of MANT-CDG. The fluorescence of MANT-CDG is sensitive to changes in the microenvironment, which helped us study its interaction with three different c-di-GMP binding proteins (a diguanylate cyclase, a phosphodiesterase, and a PilZ domain-containing protein). In addition, we have shown here that MANT-CDG can inhibit diguanylate cyclase activity; however, it is hydrolyzed by c-di-GMP specific phosphodiesterase. Taken together, our data suggest that MANT-CDG behaves like native c-di-GMP, and this study raises the possibility that MANT-CDG will be a valuable research tool for the in vitro characterization of c-di-GMP signaling factors.

A full-length bifunctional protein involved in c-di-GMP turnover is required for long-term survival under nutrient starvation in Mycobacterium smegmatis

The bacterial second messenger cyclic diguanosine monophosphate (c-di-GMP) plays an important role in a variety of cellular functions, including biofilm formation, alterations in the cell surface, host colonization and regulation of bacterial flagellar motility, which enable bacteria to survive changing environmental conditions. The cellular level of c-di-GMP is regulated by a balance between opposing activities of diguanylate cyclases (DGCs) and cognate phosphodiesterases (PDE-As). Here, we report the presence and importance of a protein, MSDGC-1 (an orthologue of Rv1354c in Mycobacterium tuberculosis), involved in c-di-GMP turnover in Mycobacterium smegmatis. MSDGC-1 is a multidomain protein, having GAF, GGDEF and EAL domains arranged in tandem, and exhibits both c-di-GMP synthesis and degradation activities. Most other proteins containing GGDEF and EAL domains have been demonstrated to have either DGC or PDE-A activity. Unlike other bacteria, which harbour several copies of the protein involved in c-di-GMP turnover, M. smegmatis has a single genomic copy, deletion of which severely affects long-term survival under conditions of nutrient starvation. Overexpression of MSDGC-1 alters the colony morphology and growth profile of M. smegmatis. In order to gain insights into the regulation of the c-di-GMP level, we cloned individual domains and tested their activities. We observed a loss of activity in the separated domains, indicating the importance of full-length MSDGC-1 for controlling bifunctionality.

Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.

Markov chain modeling of evolution of strains in reinforced concrete flexural beams

From the analysis of experimentally observed variations in surface strains with loading in reinforced concrete beams, it is noted that there is a need to consider the evolution of strains (with loading) as a stochastic process. Use of Markov Chains for modeling stochastic evolution of strains with loading in reinforced concrete flexural beams is studied in this paper. A simple, yet practically useful, bi-level homogeneous Gaussian Markov Chain (BLHGMC) model is proposed for determining the state of strain in reinforced concrete beams. The BLHGMC model will be useful for predicting behavior/response of reinforced concrete beams leading to more rational design.

Methodology for Pavement Design Reliability and Back Analysis Using Markov Chain Monte Carlo Simulation

Given the increasing cost of designing and building new highway pavements, reliability analysis has become vital to ensure that a given pavement performs as expected in the field. Recognizing the importance of failure analysis to safety, reliability, performance, and economy, back analysis has been employed in various engineering applications to evaluate the inherent uncertainties of the design and analysis. The probabilistic back analysis method formulated on Bayes' theorem and solved using the Markov chain Monte Carlo simulation method with a Metropolis-Hastings algorithm has proved to be highly efficient to address this issue. It is also quite flexible and is applicable to any type of prior information. In this paper, this method has been used to back-analyze the parameters that influence the pavement life and to consider the uncertainty of the mechanistic-empirical pavement design model. The load-induced pavement structural responses (e.g., stresses, strains, and deflections) used to predict the pavement life are estimated using the response surface methodology model developed based on the results of linear elastic analysis. The failure criteria adopted for the analysis were based on the factor of safety (FOS), and the study was carried out for different sample sizes and jumping distributions to estimate the most robust posterior statistics. From the posterior statistics of the case considered, it was observed that after approximately 150 million standard axle load repetitions, the mean values of the pavement properties decrease as expected, with a significant decrease in the values of the elastic moduli of the expected layers. An analysis of the posterior statistics indicated that the parameters that contribute significantly to the pavement failure were the moduli of the base and surface layer, which is consistent with the findings from other studies. After the back analysis, the base modulus parameters show a significant decrease of 15.8% and the surface layer modulus a decrease of 3.12% in the mean value. The usefulness of the back analysis methodology is further highlighted by estimating the design parameters for specified values of the factor of safety. The analysis revealed that for the pavement section considered, a reliability of 89% and 94% can be achieved by adopting FOS values of 1.5 and 2, respectively. The methodology proposed can therefore be effectively used to identify the parameters that are critical to pavement failure in the design of pavements for specified levels of reliability. DOI: 10.1061/(ASCE)TE.1943-5436.0000455. (C) 2013 American Society of Civil Engineers.

Non-Stationary Semi-Markov Decision Processes on a Finite Horizon

We introduce and study a class of non-stationary semi-Markov decision processes on a finite horizon. By constructing an equivalent Markov decision process, we establish the existence of a piecewise open loop relaxed control which is optimal for the finite horizon problem.

A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.

A Markov Decision Theoretic Approach to Pilot Allocation and Receive Antenna Selection

This paper considers antenna selection (AS) at a receiver equipped with multiple antenna elements but only a single radio frequency chain for packet reception. As information about the channel state is acquired using training symbols (pilots), the receiver makes its AS decisions based on noisy channel estimates. Additional information that can be exploited for AS includes the time-correlation of the wireless channel and the results of the link-layer error checks upon receiving the data packets. In this scenario, the task of the receiver is to sequentially select (a) the pilot symbol allocation, i.e., how to distribute the available pilot symbols among the antenna elements, for channel estimation on each of the receive antennas; and (b) the antenna to be used for data packet reception. The goal is to maximize the expected throughput, based on the past history of allocation and selection decisions, and the corresponding noisy channel estimates and error check results. Since the channel state is only partially observed through the noisy pilots and the error checks, the joint problem of pilot allocation and AS is modeled as a partially observed Markov decision process (POMDP). The solution to the POMDP yields the policy that maximizes the long-term expected throughput. Using the Finite State Markov Chain (FSMC) model for the wireless channel, the performance of the POMDP solution is compared with that of other existing schemes, and it is illustrated through numerical evaluation that the POMDP solution significantly outperforms them.

Synthesis and evaluation of 2,5-di(4-aryloylaryloxymethyl)-1,3,4-oxadiazoles as anti-cancer agents

A series of 2,5-di(4-aryloylaryloxymethyl)-1,3,4-oxadiazoles 9a-j were obtained via multistep synthesis from hydroxybenzophenones 4a-e. The cytotoxicity of compounds 9a-j was evaluated against human leukemia cell lilies (K562 and CEM). The compounds exhibited moderate to good anti-cancer activity with compounds 9b and 9i having a chloro group exhibiting the best activity (IC50 = 10 mu M). Compound 9i exhibited activity against both the cell lines and 9b only exhibited activity against CEM. Further, a lactate dehydrogenase (LDH) assay and DNA fragmentation studies of the compounds 9a-j were also performed. (C) 2013 Elsevier Masson SAS. All rights reserved.

Identification and characterization of starvation induced msdgc-1 promoter involved in the c-di-GMP turnover

C-di-GMP Bis-(3'-5')-cyclic-dimeric-guanosine monophosphate], a second messenger is involved in intracellular communication in the bacterial species. As a result several multi-cellular behaviors in both Gram-positive and Gram-negative bacteria are directly linked to the intracellular level of c-di-GMP. The cellular concentration of c-di-GMP is maintained by two opposing activities, diguanylate cyclase (DGC) and phosphodiesterase (PDE-A). In Mycobacterium smegmatis, a single bifunctional protein MSDGC-1 is responsible for the cellular concentration of c-di-GMP. A better understanding of the regulation of c-di-GMP at the genetic level is necessary to control the function of above two activities. In this work, we have characterized the promoter element present in msdgc-1 along with the + 1 transcription start site and identified the sigma factors that regulate the transcription of msdgc-1. Interestingly, msdgc-1 utilizes SigA during the initial phase of growth, whereas near the stationary phase SigB containing RNA polymerase takes over the expression of msdgc-1. We report here that the promoter activity of msdgc-1 increases during starvation or depletion of carbon source like glucose or glycerol. When msdgc-1 is deleted, the numbers of viable cells are similar to 10 times higher in the stationary phase in comparison to that of the wild type. We propose here that msdgc-1 is involved in the regulation of cell population density. (C) 2013 Elsevier B.V. All rights reserved.

Backbone-modified amphiphilic cyclic di- and tetrasaccharides

Synthesis of amphiphilic, cyclic di- and tetrasaccharides, which incorporate a methylene moiety at the inter-glycosidic bond, is reported. The amphiphilic properties of the new cyclic tetrasaccharide host were identified through assessing the solubilities of guests in aqueous and in organic solvents. The glycosidic bond stability of the cyclic tetrasaccharide under aqueous acidic condition was also verified.

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

An Unusual Ring-Contraction/Rearrangement Sequence for Making Functionalized Di- and Triquinanes

A novel ring contraction/rearrangement sequence leading to functionalized 2,8-oxymethano-bridged di- and triquinane compounds is observed in the reaction of various substituted 1-methyl-4-isopropenyl-6-oxabicylo3.2.1]octan-8-ones with Lewis acids. The reaction is novel and is unprecedented for the synthesis of di- and triquinane frameworks.