Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.

Analytical model for congestion control and throughput with TCP CUBIC connections

We develop a Markov model for a TCP CUBIC connection. Next we use it to obtain approximate expressions for throughput when there may be queuing in the network. Finally we provide the throughputs different TCP CUBIC and TCP NewReno connections obtain while sharing a channel when they may have different round trip delays and packet loss probabilities.

New insights from a fixed-point analysis of single cell IEEE 802.11 WLANs

We study a fixed-point formalization of the well-known analysis of Bianchi. We provide a significant simplification and generalization of the analysis. In this more general framework, the fixed-point solution and performance measures resulting from it are studied. Uniqueness of the fixed point is established. Simple and general throughput formulas are provided. It is shown that the throughput of any flow will be bounded by the one with the smallest transmission rate. The aggregate throughput is bounded by the reciprocal of the harmonic mean of the transmission rates. In an asymptotic regime with a large number of nodes, explicit formulas for the collision probability, the aggregate attempt rate, and the aggregate throughput are provided. The results from the analysis are compared with ns2 simulations and also with an exact Markov model of the backoff process. It is shown how the saturated network analysis can be used to obtain TCP transfer throughputs in some cases.

Delay optimal control algorithm for a multiaccess fading channel with peak power constraint

We consider an optimal power and rate scheduling problem for a multiaccess fading wireless channel with the objective of minimising a weighted sum of mean packet transmission delay subject to a peak power constraint. The base station acts as a controller which, depending upon the buffer lengths and the channel state of each user, allocates transmission rate and power to individual users. We assume perfect channel state information at the transmitter and the receiver. We also assume a Markov model for the fading and packet arrival processes. The policy obtained represents a form of Indexability.

Computation-Based Reliability Analysis in Real Time Applications

Reliability analysis for computing systems in aerospace applications must account for actual computations the system performs in the use environment. This paper introduces a theoretical nonhomogeneous Markov model for such applications.

Effect of dynamical asymmetry on the viscosity of a random copolymer melt

The variation of the viscosity as a function of the sequence distribution in an A-B random copolymer melt is determined. The parameters that characterize the random copolymer are the fraction of A monomers f, the parameter lambda which determines the correlation in the monomer identities along a chain and the Flory chi parameter chi(F) which determines the strength of the enthalpic repulsion between monomers of type A and B. For lambda>0, there is a greater probability of finding like monomers at adjacent positions along the chain, and for lambda<0 unlike monomers are more likely to be adjacent to each other. The traditional Markov model for the random copolymer melt is altered to remove ultraviolet divergences in the equations for the renormalized viscosity, and the phase diagram for the modified model has a binary fluid type transition for lambda>0 and does not exhibit a phase transition for lambda<0. A mode coupling analysis is used to determine the renormalization of the viscosity due to the dependence of the bare viscosity on the local concentration field. Due to the dissipative nature of the coupling. there are nonlinearities both in the transport equation and in the noise correlation. The concentration dependence of the transport coefficient presents additional difficulties in the formulation due to the Ito-Stratonovich dilemma, and there is some ambiguity about the choice of the concentration to be used while calculating the noise correlation. In the Appendix, it is shown using a diagrammatic perturbation analysis that the Ito prescription for the calculation of the transport coefficient, when coupled with a causal discretization scheme, provides a consistent formulation that satisfies stationarity and the fluctuation dissipation theorem. This functional integral formalism is used in the present analysis, and consistency is verified for the present problem as well. The upper critical dimension for this type of renormaliaation is 2, and so there is no divergence in the viscosity in the vicinity of a critical point. The results indicate that there is a systematic dependence of the viscosity on lambda and chi(F). The fluctuations tend to increase the viscosity for lambda<0, and decrease the viscosity for lambda>0, and an increase in chi(F) tends to decrease the viscosity. (C) 1996 American Institute of Physics.

Link failure detection for maintaining session continuity in packet data networks

A link failure in the path of a virtual circuit in a packet data network will lead to premature disconnection of the circuit by the end-points. A soft failure will result in degraded throughput over the virtual circuit. If these failures can be detected quickly and reliably, then appropriate rerouteing strategies can automatically reroute the virtual circuits that use the failed facility. In this paper, we develop a methodology for analysing and designing failure detection schemes for digital facilities. Based on errored second data, we develop a Markov model for the error and failure behaviour of a T1 trunk. The performance of a detection scheme is characterized by its false alarm probability and the detection delay. Using the Markov model, we analyse the performance of detection schemes that use physical layer or link layer information. The schemes basically rely upon detecting the occurrence of severely errored seconds (SESs). A failure is declared when a counter, that is driven by the occurrence of SESs, reaches a certain threshold.For hard failures, the design problem reduces to a proper choice;of the threshold at which failure is declared, and on the connection reattempt parameters of the virtual circuit end-point session recovery procedures. For soft failures, the performance of a detection scheme depends, in addition, on how long and how frequent the error bursts are in a given failure mode. We also propose and analyse a novel Level 2 detection scheme that relies only upon anomalies observable at Level 2, i.e. CRC failures and idle-fill flag errors. Our results suggest that Level 2 schemes that perform as well as Level 1 schemes are possible.

Throughput analysis of primary and secondary networks in a shared IEEE 802.11 system

In this paper, we analyze the coexistence of a primary and a secondary (cognitive) network when both networks use the IEEE 802.11 based distributed coordination function for medium access control. Specifically, we consider the problem of channel capture by a secondary network that uses spectrum sensing to determine the availability of the channel, and its impact on the primary throughput. We integrate the notion of transmission slots in Bianchi's Markov model with the physical time slots, to derive the transmission probability of the secondary network as a function of its scan duration. This is used to obtain analytical expressions for the throughput achievable by the primary and secondary networks. Our analysis considers both saturated and unsaturated networks. By performing a numerical search, the secondary network parameters are selected to maximize its throughput for a given level of protection of the primary network throughput. The theoretical expressions are validated using extensive simulations carried out in the Network Simulator 2. Our results provide critical insights into the performance and robustness of different schemes for medium access by the secondary network. In particular, we find that the channel captures by the secondary network does not significantly impact the primary throughput, and that simply increasing the secondary contention window size is only marginally inferior to silent-period based methods in terms of its throughput performance.

Experimental determination and theoretical correlation for the solubilities of dicarboxylic acid esters in supercritical carbon dioxide

The synthesis of high molecular weight esters such as bis (2-ethylhexyl) sebacate is of significance for its use as a lubricant. This ester is synthesized by the transesterification of dimethyl sebacate with 2-ethylhexanol. Therefore, the solubilities of bis (2-ethylhexyl) sebacate and dimethyl sebacate were determined at 308-328 K at pressures of 10-18 MPa in supercritical carbon dioxide. The solubility of dimethyl sebacate was always higher than bis (2-ethylhexyl) sebacate at a given temperature and pressure. The Mendez-Teja model was used to verify the self-consistency of data. Further, a new semi-empirical model with three parameters was developed using the solution theory coupled with Wilson activity coefficient. This model was used to correlate the experimental data of this work and solubilities of many high molecular weight esters reported in the literature. (C) 2015 Elsevier B.V. All rights reserved.

Semi-active vibration isolation using a near-zero-spring-rate device:steady state analysis of a single-degree-of-freedom model

A vibration isolator is described which incorporates a near-zero-spring-rate device within its operating range. The device is an assembly of a vertical spring in parallel with two inclined springs. A low spring rate is achieved by combining the equivalent stiffness in the vertical direction of the inclined springs with the stiffness of the vertical central spring. It is shown that there is a relation between the geometry and the stiffness of the individual springs that results in a low spring rate. Computer simulation studies of a single-degree-of-freedom model for harmonic base input show that the performance of the proposed scheme is superior to that of the passive schemes with linear springs and skyhook damping configuration. The response curves show that, for small to large amplitudes of base disturbance, the system goes into resonance at low frequencies of excitation. Thus, it is possible to achieve very good isolation over a wide low-frequency band. Also, the damper force requirements for the proposed scheme are much lower than for the damper force of a skyhook configuration or a conventional linear spring with a semi-active damper.

Markov Chain Model for Cracking Behavior of Reinforced Concrete Beams

In this paper, nonhomogeneous Markov chains are proposed for modeling the cracking behavior of reinforced concrete beams subjected to monotonically increasing loads. The model facilitates prediction of the maximum crackwidth at a given load given the crackwidth at a lower load level, and thus leads to a better understanding of the cracking phenomenon. To illustrate the methodology developed, the results of three reinforced concrete beams tested in the laboratory are analyzed and presented.

Multiscale stochastic approximation for parametric optimization of hidden Markov models

A two-time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to ''infinitesimal perturbation analysis.'' Its convergence is analyzed, and a queueing example is presented.

Causality analysis of groundwater dynamics based on a Vector Autoregressive model in the semi-arid basin of Gundal (South India)

Causal relationships existing between observed levels of groundwater in a semi-arid sub-basin of the Kabini River basin (Karnataka state, India) are investigated in this study. A Vector Auto Regressive model is used for this purpose. Its structure is built on an upstream/downstream interaction network based on observed hydro-physical properties. Exogenous climatic forcing is used as an input based on cumulated rainfall departure. Optimal models are obtained thanks to a trial approach and are used as a proxy of the dynamics to derive causal networks. It appears to be an interesting tool for analysing the causal relationships existing inside the basin. The causal network reveals 3 main regions: the Northeastern part of the Gundal basin is closely coupled to the outlet dynamics. The Northwestern part is mainly controlled by the climatic forcing and only marginally linked to the outlet dynamic. Finally, the upper part of the basin plays as a forcing rather than a coupling with the lower part of the basin allowing for a separate analysis of this local behaviour. The analysis also reveals differential time scales at work inside the basin when comparing upstream oriented with downstream oriented causalities. In the upper part of the basin, time delays are close to 2 months in the upward direction and lower than 1 month in the downward direction. These time scales are likely to be good indicators of the hydraulic response time of the basin which is a parameter usually difficult to estimate practically. This suggests that, at the sub-basin scale, intra-annual time scales would be more relevant scales for analysing or modelling tropical basin dynamics in hard rock (granitic and gneissic) aquifers ubiquitous in south India. (c) 2012 Elsevier B.V. All rights reserved.

Semi-analytical finite element analysis of a strip footing on an elastic reinforced soil

A plane strain elastic interaction analysis of a strip footing resting on a reinforced soil bed has been made by using a combined analytical and finite element method (FEM). In this approach the stiffness matrix for the footing has been obtained using the FEM, For the reinforced soil bed (halfplane) the stiffness matrix has been obtained using an analytical solution. For the latter, the reinforced zone has been idealised as (i) an equivalent orthotropic infinite strip (composite approach) and (ii) a multilayered system (discrete approach). In the analysis, the interface between the strip footing and reinforced halfplane has been assumed as (i) frictionless and (ii) fully bonded. The contact pressure distribution and the settlement reduction have been given for different depths of footing and scheme of reinforcement in soil. The load-deformation behaviour of the reinforced soil obtained using the above modelling has been compared with some available analytical and model test results. The equivalent orthotropic approach proposed in this paper is easy to program and is shown to predict the reinforcing effects reasonably well.

An analytical model for performance evaluation of multimedia applications over EDCA in an IEEE 802.11e WLAN

We extend the modeling heuristic of (Harsha et al. 2006. In IEEE IWQoS 06, pp 178 - 187) to evaluate the performance of an IEEE 802.11e infrastructure network carrying packet telephone calls, streaming video sessions and TCP controlled file downloads, using Enhanced Distributed Channel Access (EDCA). We identify the time boundaries of activities on the channel (called channel slot boundaries) and derive a Markov Renewal Process of the contending nodes on these epochs. This is achieved by the use of attempt probabilities of the contending nodes as those obtained from the saturation fixed point analysis of (Ramaiyan et al. 2005. In Proceedings ACM Sigmetrics, `05. Journal version accepted for publication in IEEE TON). Regenerative analysis on this MRP yields the desired steady state performance measures. We then use the MRP model to develop an effective bandwidth approach for obtaining a bound on the size of the buffer required at the video queue of the AP, such that the streaming video packet loss probability is kept to less than 1%. The results obtained match well with simulations using the network simulator, ns-2. We find that, with the default IEEE 802.11e EDCA parameters for access categories AC 1, AC 2 and AC 3, the voice call capacity decreases if even one streaming video session and one TCP file download are initiated by some wireless station. Subsequently, reducing the voice calls increases the video downlink stream throughput by 0.38 Mbps and file download capacity by 0.14 Mbps, for every voice call (for the 11 Mbps PHY). We find that a buffer size of 75KB is sufficient to ensure that the video packet loss probability at the QAP is within 1%.