Multiscale stochastic approximation for parametric optimization of hidden Markov models

A two-time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to ''infinitesimal perturbation analysis.'' Its convergence is analyzed, and a queueing example is presented.

The actor-critic algorithm as multi-time-scale stochastic approximation

The actor-critic algorithm of Barto and others for simulation-based optimization of Markov decision processes is cast as a two time Scale stochastic approximation. Convergence analysis, approximation issues and an example are studied.

Optimal control of a stochastic hybrid system with discounted cost

We address the optimal control problem of a very general stochastic hybrid system with both autonomous and impulsive jumps. The planning horizon is infinite and we use the discounted-cost criterion for performance evaluation. Under certain assumptions, we show the existence of an optimal control. We then derive the quasivariational inequalities satisfied by the value function and establish well-posedness. Finally, we prove the usual verification theorem of dynamic programming.

Sex-specific organisation of middle repetitive DNA sequences in the mealybug Planococcus lilacinus

Differential organisation of homologous chromosomes is related to both sex determination and genomic imprinting in coccid insects, the mealybugs. We report here the identification of two middle repetitive sequences that are differentially organised between the two sexes and also within the same diploid nucleus. These two sequences form a part of the male-specific nuclease-resistant chromatin (NRC) fraction of a mealybug Planococcus lilacinus. To understand the phenomenon of differential organisation we have analysed the components of NRC by cloning the DNA sequences present, deciphering their primary sequence, nucleosomal organisation, genomic distribution and cytological localisation, Our observations suggest that the middle repetitive sequences within NRC are functionally significant and we discuss their probable involvement in male-specific chromatin organisation.

Nonlinear Dynamics of Beams with Stochastic Parameter Variations

The aim of this paper is to investigate the steady state response of beams under the action of random support motions. The study is of relevance in the context of earthquake response of extended land based structures such as pipelines and long span bridges, and, secondary systems such as piping networks in nuclear power plant installations. The following complicating features are accounted for in the response analysis: (a) differential support motions: this is characterized in terms of cross power spectral density functions associated with distinct support motions, (b) nonlinear support conditions, and (c) stochastically inhomogeneous stiffness and mass variations of the beam structure; questions on non-Gaussian models for these variations are considered. The method of stochastic finite elements is combined with equivalent linearization technique and Monte Carlo simulations to obtain response moments.

The Borkar-Meyn theorem for asynchronous stochastic approximations

In this paper, we give a generalization of a result by Borkar and Meyn (2000) 1], on the stability and convergence of synchronous-update stochastic approximation algorithms, to the case of asynchronous stochastic approximations with delays. We then describe an interesting application of the result to asynchronous distributed temporal difference (TD) learning with function approximation and delays. (C) 2011 Elsevier B.V. All rights reserved.

Modeling of nonreacting and reacting turbulent spray jets using a fully stochastic separated flow approach

Numerical modeling of several turbulent nonreacting and reacting spray jets is carried out using a fully stochastic separated flow (FSSF) approach. As is widely used, the carrier-phase is considered in an Eulerian framework, while the dispersed phase is tracked in a Lagrangian framework following the stochastic separated flow (SSF) model. Various interactions between the two phases are taken into account by means of two-way coupling. Spray evaporation is described using a thermal model with an infinite conductivity in the liquid phase. The gas-phase turbulence terms are closed using the k-epsilon model. A novel mixture fraction based approach is used to stochastically model the fluctuating temperature and composition in the gas phase and these are then used to refine the estimates of the heat and mass transfer rates between the droplets and the surrounding gas-phase. In classical SSF (CSSF) methods, stochastic fluctuations of only the gas-phase velocity are modeled. Successful implementation of the FSSF approach to turbulent nonreacting and reacting spray jets is demonstrated. Results are compared against experimental measurements as well as with predictions using the CSSF approach for both nonreacting and reacting spray jets. The FSSF approach shows little difference from the CSSF predictions for nonreacting spray jets but differences are significant for reacting spray jets. In general, the FSSF approach gives good predictions of the flame length and structure but further improvements in modeling may be needed to improve the accuracy of some details of the Predictions. (C) 2011 The Combustion Institute. Published by Elsevier Inc. All rights reserved.

Low correlation interleaved QAM sequences

A low correlation interleaved QAM sequence family is presented here. In a CDMA setting, these sequences have the ability to transport a large amount of data as well as enable variable-rate signaling on the reverse link. The new interleaved selected family INQ has period N, normalized maximum correlation parameter thetasmacrmax bounded above by lsim a radicN, where a ranges from 1.17 in the 16-QAM case to 1.99 for large M2-QAM, where M = 2m, m ges 2. Each user is enabled to transfer m + 1 bits of data per period of the spreading sequence. These constructions have the lowest known value of maximum correlation of any sequence family with the same alphabet.

Doubly spectral stochastic finite element method (DSSFEM) for random field problems

Uncertainties in complex dynamic systems play an important role in the prediction of a dynamic response in the mid- and high-frequency ranges. For distributed parameter systems, parametric uncertainties can be represented by random fields leading to stochastic partial differential equations. Over the past two decades, the spectral stochastic finite-element method has been developed to discretize the random fields and solve such problems. On the other hand, for deterministic distributed parameter linear dynamic systems, the spectral finite-element method has been developed to efficiently solve the problem in the frequency domain. In spite of the fact that both approaches use spectral decomposition (one for the random fields and the other for the dynamic displacement fields), very little overlap between them has been reported in literature. In this paper, these two spectral techniques are unified with the aim that the unified approach would outperform any of the spectral methods considered on their own. An exponential autocorrelation function for the random fields, a frequency-dependent stochastic element stiffness, and mass matrices are derived for the axial and bending vibration of rods. Closed-form exact expressions are derived by using the Karhunen-Loève expansion. Numerical examples are given to illustrate the unified spectral approach.

Pruned Universal Symbol Sequences for LZW based Language identification

We present a improved language modeling technique for Lempel-Ziv-Welch (LZW) based LID scheme. The previous approach to LID using LZW algorithm prepares the language pattern table using LZW algorithm. Because of the sequential nature of the LZW algorithm, several language specific patterns of the language were missing in the pattern table. To overcome this, we build a universal pattern table, which contains all patterns of different length. For each language it's corresponding language specific pattern table is constructed by retaining the patterns of the universal table whose frequency of appearance in the training data is above the threshold.This approach reduces the classification score (Compression Ratio [LZW-CR] or the weighted discriminant score[LZW-WDS]) for non native languages and increases the LID performance considerably.

Sequences for Phase-Encoded Optical CDMA

In phase-encoded optical CDMA (OCDMA) spreading is achieved by encoding the phase of signal spectrum. Here, a mathematical model for the output signal of a phase-encoded OCDMA system is first derived. This is shown to lead to a performance metric for the design of spreading sequences for asynchronous transmission. Generalized bent functions are used to construct a family of efficient phase-encoding sequences. It is shown how M-ary modulation of these spreading sequences is possible. The problem of designing efficient phaseencoded sequences is then related to the problem of minimizing PMEPR (peak-to-mean envelope power ratio) in an OFDM communication system.

Optimality of orthogonal sequences with multi-dimensional signaling

The sum capacity on a symbol-synchronous CDMA system having processing gain N and supporting K power constrained users is achieved by employing any set of N orthogonal sequences if a few users are allowed to signal along multiple dimensions. Analogously, the minimum received power (energy-per-chip) on the symbolsynchronous CDMA system supporting K users that demand specified data rates is attained by employing any set of N orthogonal sequences. At most (N - 1) users need to be split and if there are no oversized users, these split users need to signal only in two dimensions each. These results show that sum capacity or minimum sum power can be achieved with minimal downlink signaling.