Stochastic stability of nonlinear discrete-time Markovian jump systems with time-varying delay and partially unknown transition rates

This paper is concerned with stochastic stability of a class of nonlinear discrete-time Markovian jump systems with interval time-varying delay and partially unknown transition probabilities. A new weighted summation inequality is first derived. We then employ the newly derived inequality to establish delay-dependent conditions which guarantee the stochastic stability of the system. These conditions are derived in terms of tractable matrix inequalities which can be computationally solved by various convex optimized algorithms. Numerical examples are provided to illustrate the effectiveness of the obtained results.

Modelling Spikes in Electricity Prices

uring periods of market stress, electricity prices can rise dramatically. Electricity retailers cannot pass these extreme prices on to customers because of retail price regulation. Improved prediction of these price spikes therefore is important for risk management. This paper builds a time-varying-probability Markov-switching model of Queensland electricity prices, aimed particularly at forecasting price spikes. Variables capturing demand and weather patterns are used to drive the transition probabilities. Unlike traditional Markov-switching models that assume normality of the prices in each state, the model presented here uses a generalised beta distribution to allow for the skewness in the distribution of electricity prices during high-price episodes.

Business cycle dynamics with duration dependence and leading indicators

Durland and McCurdy [Durland, J.M., McCurdy, T.H., 1994. Duration-dependent transitions in a Markov model of US GNP growth. Journal of Business and Economic Statistics 12, 279–288] investigated the issue of duration dependence in US business cycle phases using a Markov regime-switching approach, introduced by Hamilton [Hamilton, J., 1989. A new approach to the analysis of time series and the business cycle. Econometrica 57, 357–384] and extended to the case of variable transition parameters by Filardo [Filardo, A.J., 1994. Business cycle phases and their transitional dynamics. Journal of Business and Economic Statistics 12, 299–308]. In Durland and McCurdy’s model duration alone was used as an explanatory variable of the transition probabilities. They found that recessions were duration dependent whilst expansions were not. In this paper, we explicitly incorporate the widely-accepted US business cycle phase change dates as determined by the NBER, and use a state-dependent multinomial Logit modelling framework. The model incorporates both duration and movements in two leading indexes – one designed to have a short lead (SLI) and the other designed to have a longer lead (LLI) – as potential explanatory variables. We find that doing so suggests that current duration is not only a significant determinant of transition out of recessions, but that there is some evidence that it is also weakly significant in the case of expansions. Furthermore, we find that SLI has more informational content for the termination of recessions whilst LLI does so for expansions.

Optimistic linear programming gives logarithmic regret for irreducible MDPs

We present an algorithm called Optimistic Linear Programming (OLP) for learning to optimize average reward in an irreducible but otherwise unknown Markov decision process (MDP). OLP uses its experience so far to estimate the MDP. It chooses actions by optimistically maximizing estimated future rewards over a set of next-state transition probabilities that are close to the estimates, a computation that corresponds to solving linear programs. We show that the total expected reward obtained by OLP up to time T is within C(P) log T of the reward obtained by the optimal policy, where C(P) is an explicit, MDP-dependent constant. OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities, the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm. OLP is also similar in flavor to an algorithm recently proposed by Auer and Ortner. But OLP is simpler and its regret bound has a better dependence on the size of the MDP.

A Bayesian‐Markov process for reliability prediction

Accurate reliability prediction for large-scale, long lived engineering is a crucial foundation for effective asset risk management and optimal maintenance decision making. However, a lack of failure data for assets that fail infrequently, and changing operational conditions over long periods of time, make accurate reliability prediction for such assets very challenging. To address this issue, we present a Bayesian-Marko best approach to reliability prediction using prior knowledge and condition monitoring data. In this approach, the Bayesian theory is used to incorporate prior information about failure probabilities and current information about asset health to make statistical inferences, while Markov chains are used to update and predict the health of assets based on condition monitoring data. The prior information can be supplied by domain experts, extracted from previous comparable cases or derived from basic engineering principles. Our approach differs from existing hybrid Bayesian models which are normally used to update the parameter estimation of a given distribution such as the Weibull-Bayesian distribution or the transition probabilities of a Markov chain. Instead, our new approach can be used to update predictions of failure probabilities when failure data are sparse or nonexistent, as is often the case for large-scale long-lived engineering assets.

A probabilistic framework to monitor a multi-mode outdoor robot

This paper presents an approach to autonomously monitor the behavior of a robot endowed with several navigation and locomotion modes, adapted to the terrain to traverse. The mode selection process is done in two steps: the best suited mode is firstly selected on the basis of initial information or a qualitative map built on-line by the robot. Then, the motions of the robot are monitored by various processes that update mode transition probabilities in a Markov system. The paper focuses on this latter selection process: the overall approach is depicted, and preliminary experimental results are presented

Output feedback control of networked systems with a stochastic communication protocol

This paper addresses an output feedback control problem for a class of networked control systems (NCSs) with a stochastic communication protocol. Under the scenario that only one sensor is allowed to obtain the communication access at each transmission instant, a stochastic communication protocol is first defined, where the communication access is modelled by a discrete-time Markov chain with partly unknown transition probabilities. Secondly, by use of a network-based output feedback control strategy and a time-delay division method, the closed-loop system is modeled as a stochastic system with multi time-varying delays, where the inherent characteristic of the network delay is well considered to improve the control performance. Then, based on the above constructed stochastic model, two sufficient conditions are derived for ensuring the mean-square stability and stabilization of the system under consideration. Finally, two examples are given to show the effectiveness of the proposed method.

trans gene regulation in adaptive evolution: A genetic algorithm model

This is a continuation of earlier studies on the evolution of infinite populations of haploid genotypes within a genetic algorithm framework. We had previously explored the evolutionary consequences of the existence of indeterminate—“plastic”—loci, where a plastic locus had a finite probability in each generation of functioning (being switched “on”) or not functioning (being switched “off”). The relative probabilities of the two outcomes were assigned on a stochastic basis. The present paper examines what happens when the transition probabilities are biased by the presence of regulatory genes. We find that under certain conditions regulatory genes can improve the adaptation of the population and speed up the rate of evolution (on occasion at the cost of lowering the degree of adaptation). Also, the existence of regulatory loci potentiates selection in favour of plasticity. There is a synergistic effect of regulatory genes on plastic alleles: the frequency of such alleles increases when regulatory loci are present. Thus, phenotypic selection alone can be a potentiating factor in a favour of better adaptation.

Spectroscopic investigations and luminescence spectra of Nd3+ and Dy3+ doped different phosphate glasses

Spectral properties of Nd3+ and Dy3+ ions in different phosphate glasses were studied and several spectroscopic parameters were reported. Covalency of rare-earth-oxygen bond was studied in these phosphate glass matrices with the variation of modifier in host glass matrix Using Judd-Ofelt intensity parameters (Omega(2), Omega(4) and Omega(6)), radiative transition probabilities (A) and radiative lifetimes (tau(R)) of certain excited states of Nd3+ and Dy3+ ions are estimated in these glass matrices. From the magnitudes of branching ratios (beta(R)) and integrated absorption cross-sections (Sigma), certain transitions of both the ions are identified for laser excitation. From the emission spectra, peak stimulated emission cross-sections (sigma(P)) are evaluated for the emission transitions observed in all these phosphate glass matrices for both Nd3+ and Dy3+ ions. (C) 2009 Elsevier B.V. All rights reserved.

Temperature- and pressure-dependent study of Cl-35 NQR frequency and spin lattice relaxation time in 2,3-dichloroanisole

The temperature and pressure dependence of Cl-35 NQR frequency and spin lattice relaxation time (T-1) were investigated in 2,3-dichloroanisole. Two NQR signals were observed throughout the temperature and pressure range studied. T-1 were measured in the temperature range from 77 to 300 K and from atmospheric pressure to 5 kbar. Relaxation was found to be due to the torsional motion of the molecule and also reorientation f motion of the CH3 group. T-1 versus temperature data were analyzed on the basis of Woessner and Gutowsky model, and the activation energy for the reorientation of the CH3 group was estimated. The temperature dependence of the average torsional lifetimes of the molecules and the transition probabilities were also obtained. NQR frequency shows a nonlinear behavior with pressure, indicating both dynamic and static effects of pressure. The pressure coefficients were observed to be positive for both the lines. A thermodynamic analysis of the data was carried out to determine the constant volume temperature coefficients of the NQR frequency. The variation of spin lattice time with pressure was very small, showing that the relaxation is mainly due to the torsional motions of the molecules. Copyright (C) 2010 John Wiley & Sons, Ltd.

Studies on optical absorption and photoluminescenceproperties of Pr3+ and Nd3+ doped mixed alkali chloroborate glasses

This article presents the optical absorption and emission properties of Pr3+ and Nd3+ doped two different mixed alkali chloroborate glass matrices of the type 70B(2)O(3)center dot xLiCl center dot(30 - x)NaCl and 70B(2)O(3)center dot xLiCl center dot(30 - x)KCl (x = 5, 10, 15.20 and 25). The variation of Judd-Ofelt parameters (Omega(2), Omega(4) and Omega(6)), total radiative transition probabilities (A(T)), radiative lifetimes (tau(R)) and emission cross-sections (sigma(p)) with the variation of alkali contents in the glass matrix have been discussed in detail. The changes in the peak wavelengths of the hypersensitive transition and intensity parameters with x are correlated to the structural changes in the host matrix. The estimated radiative lifetimes of certain excited states of Pr3+ and Nd3+ in these two glass matrices are reported. Peak stimulated emission cross-sections (sigma(p)) are reported for the observed emission transitions of Pr3+ and Nd3+ ions. Branching ratios (beta) of the observed emission transitions obtained from the Judd-Ofelt theory are compared with the values obtained from the emission spectra. (C) 2010 Elsevier B.V. All rights reserved.

Dipolar cross-correlation effects in the nuclear overhauser experiments of weakly and strongly coupled spins

The effect of dipolar cross correlation in 1H---1H nuclear Overhauser effect experiments is investigated by detailed calculation in an ABX spin system. It is found that in weakly coupled spin systems, the cross-correlation effects are limited to single-quantum transition probabilities and decrease in magnitude as ωτc increases. Strong coupling, however, mixes the states and the cross correlations affect the zero-quantum and double-quantum transition probabilities as well. The effect of cross correlation in steady-state and transient NOE experiments is studied as a function of strong coupling and ωτc. The results for steady-state NOE experiments are calculated analytically and those for transient NOE experiments are calculated numerically. The NOE values for the A and B spins have been calculated by assuming nonselective perturbation of all the transitions of the X spin. A significant effect of cross correlation is found in transient NOE experiments of weakly as well as strongly coupled spins when the multiplets are resolved. Cross correlation manifests itself largely as a multiplet effect in the transient NOE of weakly coupled spins for nonselective perturbation of all X transitions. This effect disappears for a measuring pulse of 90° or when the multiplets are not resolved. For steady-state experiments, the effect of cross correlation is analytically zero for weakly coupled spins and small for strongly coupled spins.

Monte-Carlo estimation of time-dependent statistical characteristics of random dynamical systems

The problem of estimating the time-dependent statistical characteristics of a random dynamical system is studied under two different settings. In the first, the system dynamics is governed by a differential equation parameterized by a random parameter, while in the second, this is governed by a differential equation with an underlying parameter sequence characterized by a continuous time Markov chain. We propose, for the first time in the literature, stochastic approximation algorithms for estimating various time-dependent process characteristics of the system. In particular, we provide efficient estimators for quantities such as the mean, variance and distribution of the process at any given time as well as the joint distribution and the autocorrelation coefficient at different times. A novel aspect of our approach is that we assume that information on the parameter model (i.e., its distribution in the first case and transition probabilities of the Markov chain in the second) is not available in either case. This is unlike most other work in the literature that assumes availability of such information. Also, most of the prior work in the literature is geared towards analyzing the steady-state system behavior of the random dynamical system while our focus is on analyzing the time-dependent statistical characteristics which are in general difficult to obtain. We prove the almost sure convergence of our stochastic approximation scheme in each case to the true value of the quantity being estimated. We provide a general class of strongly consistent estimators for the aforementioned statistical quantities with regular sample average estimators being a specific instance of these. We also present an application of the proposed scheme on a widely used model in population biology. Numerical experiments in this framework show that the time-dependent process characteristics as obtained using our algorithm in each case exhibit excellent agreement with exact results. (C) 2010 Elsevier Inc. All rights reserved.

An integrated model for optimal reservoir operation for irrigation of multiple crops

An integrated model is developed, based on seasonal inputs of reservoir inflow and rainfall in the irrigated area, to determine the optimal reservoir release policies and irrigation allocations to multiple crops. The model is conceptually made up of two modules, Module 1 is an intraseasonal allocation model to maximize the sum of relative yields of all crops, for a given state of the system, using linear programming (LP). The module takes into account reservoir storage continuity, soil moisture balance, and crop root growth with time. Module 2 is a seasonal allocation model to derive the steady state reservoir operating policy using stochastic dynamic programming (SDP). Reservoir storage, seasonal inflow, and seasonal rainfall are the state variables in the SDP. The objective in SDP is to maximize the expected sum of relative yields of all crops in a year. The results of module 1 and the transition probabilities of seasonal inflow and rainfall form the input for module 2. The use of seasonal inputs coupled with the LP-SDP solution strategy in the present formulation facilitates in relaxing the limitations of an earlier study, while affecting additional improvements. The model is applied to an existing reservoir in Karnataka State, India.

The effect of host glass on optical absorption and fluorescence of Nd3+ in xNa(2)O-(30-x)K2O-70B(2)O(3) glasses

The effect of host glass composition on the optical absorption and fluorescence spectra of Nd3+ has been studied in mixed alkali borate glasses of the type xNa(2)O-(30-x)K2O-69.5B(2)O(3)-0.5Nd(2)O(3) (X = 5,10,15,20 and 25). Various spectroscopic parameters such as Racah (E-1, E-2 and E-3), spin-orbit (xi(4f)) and configuration interaction (alpha, beta) parameters have been calculated. The Judd-Ofelt intensity parameters (Omega(lambda)) have been calculated and the radiative transition probabilities (A(rad)), radiative lifetimes (tau(r)), branching ratios (beta) and integrated absorption cross sections (Sigma) have been obtained for certain excited states of the Nd3+, ion and are discussed with respect to x. From the fluorescence spectra, the effective fluorescence line widths (Deltalambda(eff)) and stimulated emission cross sections (sigma(p)) have been obtained for the three transitions F-4(3/2) --> I-4(9/2), F-4(3/2) --> I-4(11/2) and F-4(3/2) --> I-4(13/2) of Nd3+. The stimulated emission cross section (sigma(p)) values are found to be in the range (2.0-4.8) x 10(-2)0 cm(2) and they are large enough to indicate that the mixed alkali borate glasses could be potential laser host materials.